Image coding device, image decoding device, methods thereof, and programs

ABSTRACT

An image coding device including: an edge detecting section configured to perform edge detection using an image signal of a reference image for a coding object block; a transform block setting section configured to set transform blocks by dividing the coding object block such that a boundary between the blocks after division does not include an edge on a basis of a result of the edge detection; and a coding processing section configured to generate coded data by performing processing including an orthogonal transform of each of the transform blocks.

BACKGROUND

The present disclosure relates to an image coding device, an imagedecoding device, methods thereof, and programs, and particularly toenabling improvements of subjective image quality and coding efficiency.

In related art, still images and moving images have large amounts ofdata, and are thus generally coded at a time of transmission or at atime of recording onto a medium. A coding system such as an H.264/MPEG(motion picture expert group)-4 AVC (advanced video coding) (hereinafterreferred to as H.264/AVC) system or the like performs a discrete cosinetransform (hereinafter referred to as a DCT)/inverse discrete cosinetransform (hereinafter referred to as an IDCT). The DCT/IDCT is realizedby performing a one-dimensional DCT/IDCT twice in a horizontal directionand a vertical direction. On the other hand, when prediction errors ofintra-frame prediction (intra prediction) include an edge, energy can beconcentrated more by performing a DCT in a direction along the edgerather than performing a DCT in the horizontal direction and thevertical direction.

For example, Japanese Patent Laid-Open No. 2009-272727 determines adirection of performing an orthogonal transform according to a mode ofintra-frame prediction when intra-frame prediction as in the H.264/AVCsystem using spatial correlation is performed because of a strongpossibility that a prediction direction and the direction of predictionerrors are the same. By thus determining a direction of performing anorthogonal transform, higher energy concentration is achieved, andcoding efficiency is improved.

SUMMARY

When an orthogonal transform is performed, the block size of transformblocks as units of the orthogonal transform as well as the direction ofthe orthogonal transform is an important element for improving energyconcentration. In this case, when a continuous edge straddles aplurality of transform blocks, degradation such that the edge isinterrupted at block boundaries becomes conspicuous as a result ofsubsequent quantization. In addition, because a DCT has a characteristicof easily concentrating the energy of a steady signal, coding efficiencyis decreased when a large number of transform blocks include an edge.

It is accordingly desirable to provide an image coding device, an imagedecoding device, methods thereof, and programs that can improvesubjective image quality and coding efficiency.

According to a first embodiment of the present technology, there isprovided an image coding device including: an edge detecting sectionconfigured to perform edge detection using an image signal of areference image for a coding object block; a transform block settingsection configured to set transform blocks by dividing the coding objectblock such that a boundary between the blocks after division does notinclude an edge on a basis of a result of the edge detection; and acoding processing section configured to generate coded data byperforming processing including an orthogonal transform of each of thetransform blocks.

In this technology, the position of an edge and the intensity of theedge are detected using the image signal of a reference image for acoding object block. On the basis of a result of the edge detection, thecoding object block is divided, and transform blocks to be subjected toan orthogonal transform or the like are set. The transform blocks areset such that boundaries between the transform blocks as the blocksafter the division of the coding object block do not include an edge. Inaddition, the priorities of edges are determined according to theintensities of the edges, and the transform blocks are set such that anedge of high priority is not included. In detecting the edge, an imageof a coded block adjacent to the coding object block is used as areference image. In addition, the transform blocks are set with thedetected edge estimated to be continuous in the prediction direction ofan intra-frame prediction mode. In addition, a prediction mode of highcoding efficiency is selected, coding processing is performed, andinformation indicating the selected prediction mode is included in codeddata obtained by performing the coding processing. In addition,information indicating the set transform blocks is included in the codeddata. In addition, in detecting the edge, a coded image in a temporaldirection with respect to the coding object block is also used.

According to a second embodiment of the present technology, there isprovided an image coding method including: performing edge detectionusing an image signal of a reference image for a coding object block;setting transform blocks by dividing the coding object block such that aboundary between the blocks after division does not include an edge on abasis of a result of the edge detection; and generating coded data byperforming processing including an orthogonal transform of each of thetransform blocks.

According to a third embodiment of the present technology, there isprovided a program for making image coding performed on a computer, theprogram including: performing edge detection using an image signal of areference image for a coding object block; setting transform blocks bydividing the coding object block such that a boundary between the blocksafter division does not include an edge on a basis of a result of theedge detection; and generating coded data by performing processingincluding an orthogonal transform of each of the transform blocks.

According to a fourth embodiment of the present technology, there isprovided an image decoding device including: an information extractingsection configured to extract prediction mode information from codeddata; an edge detecting section configured to perform edge detectionusing an image signal of a reference image for a decoding object block;a transform block setting section configured to set transform blocks bydividing the decoding object block such that a boundary between theblocks after division does not include an edge on a basis of theprediction mode information and a result of the edge detection; and adecoding processing section configured to generate an image signal byperforming processing including an inverse orthogonal transform of eachof the transform blocks.

In this technology, prediction mode information is extracted from codeddata. In addition, an edge is detected using the image signal of areference image for a decoding object block, and the position andintensity of the edge are detected. On the basis of a result ofdetection of the edge and the extracted prediction mode information, thedecoding object block is divided, and transform blocks to be subjectedto an inverse orthogonal transform or the like are set. The transformblocks are set such that boundaries between the transform blocks as theblocks after the division of the decoding object block do not include anedge. In addition, the priorities of edges are determined according tothe intensities of the edges, and the transform blocks are set such thatthe boundaries between the transform blocks do not include an edge ofhigh priority. In detecting the edge, an image of a decoded blockadjacent to the decoding object block is used as a reference image. Inaddition, the transform blocks are set with the detected edge estimatedto be continuous in the prediction direction of an intra-frameprediction mode. In addition, the edge is detected using a decoded imagein a temporal direction with respect to the decoding object block as areference image. After the transform blocks are thus set, a decodedimage is generated by performing processing including an inverseorthogonal transform of each of the set transform blocks.

According to a fifth embodiment of the present technology, there isprovided an image decoding method including: extracting prediction modeinformation from coded data; performing edge detection using an imagesignal of a reference image for a decoding object block; settingtransform blocks by dividing the decoding object block such that aboundary between the blocks after division does not include an edge on abasis of the prediction mode information and a result of the edgedetection; and generating an image signal by performing processingincluding inverse transform processing of each of the transform blocks.

According to a sixth embodiment of the present technology, there isprovided a program for making decoding of coded data performed on acomputer, the program including: extracting prediction mode informationfrom coded data; performing edge detection using an image signal of areference image for a decoding object block; setting transform blocks bydividing the decoding object block such that a boundary between theblocks after division does not include an edge on a basis of theprediction mode information and a result of the edge detection; andgenerating an image signal by performing processing including inversetransform processing of each of the transform blocks.

Incidentally, the programs according to the embodiments of the presenttechnology are for example programs that can be provided by storagemedia provided in a computer readable format for general-purposecomputer systems capable of executing various program codes and bycommunication media, by for example storage media such as optical disks,magnetic disks, semiconductor memories, and the like or by communicationmedia such as networks and the like. Such programs are provided in acomputer readable format, whereby processing according to the programsis realized on computer systems.

According to the embodiment of the present technology, an edge isdetected using the image signal of a reference image for a coding objectblock. On the basis of a result of the edge detection, transform blocksare set by dividing the coding object block such that boundaries betweenthe blocks after the division do not include the edge. In addition,transform processing is performed for each transform block, and codeddata is generated. An image decoding device for decoding the coded datadetects the edge using the image signal of the reference image for adecoding object block. On the basis of a result of the edge detection,transform blocks are set by dividing the decoding object block such thatboundaries between the blocks after the division do not include theedge. In addition, inverse transform processing is performed for eachtransform block, and the image signal of a decoded image is generated.

It is therefore possible to prevent a continuous edge from straddling aplurality of transform blocks and thus improve subjective image quality.In addition, transform blocks not including an edge can be increased, sothat an effect of improving efficiency of energy concentration can beobtained.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram showing a configuration of an image coding device ina first embodiment;

FIGS. 2A and 2B are diagrams showing configurations of a first transformsection and a first quantizing section;

FIG. 3 is a diagram showing a configuration of a second transformsection;

FIG. 4 is a diagram showing a configuration of a second quantizingsection;

FIGS. 5A and 5B are diagrams showing configurations of a first inversetransform section and a first dequantizing section;

FIG. 6 is a diagram showing a configuration of a second dequantizingsection;

FIG. 7 is a diagram showing a configuration of a second inversetransform section;

FIGS. 8A, 8B, 8C, and 8D are diagrams showing macroblocks of intra-frameprediction in the H.264/AVC system;

FIG. 9 is a diagram of assistance in explaining positional relationbetween a sub-block and an adjacent pixel signal;

FIGS. 10A, 10B, 10C, 10D, 10E, 10F, 10G, 10H, and 10I are diagramsshowing prediction modes for 4×4 pixels in intra-frame prediction;

FIGS. 11A, 11B, and 11C are diagrams of assistance in explainingone-dimensional DCTs in a case of a prediction mode 3;

FIGS. 12A and 12B are diagrams of assistance in explainingone-dimensional DCTs in a case of a prediction mode 5;

FIG. 13 is a flowchart (1/2) showing operation of the image codingdevice in the first embodiment;

FIG. 14 is a flowchart (2/2) showing the operation of the image codingdevice in the first embodiment;

FIGS. 15A, 15B, 15C, 15D, 15E, and 15F are diagrams of assistance inexplaining operation of a reference image edge detecting section;

FIGS. 16A, 16B, 16C, and 16D are diagrams of assistance in explainingoperation of a transform block setting section;

FIG. 17 is a flowchart showing a procedure for setting transform blocks;

FIGS. 18A, 18B, 18C, 18D, and 18E are diagrams showing a case in whichtransform blocks are set by dividing a sub-block of 8×8 pixels into fourparts;

FIG. 19 is a diagram showing a configuration of an image decoding devicein the first embodiment;

FIG. 20 is a flowchart showing operation of the image decoding device inthe first embodiment;

FIG. 21 is a diagram showing a configuration of an image coding devicein a second embodiment;

FIG. 22 is a diagram showing a configuration of an image decoding devicein the second embodiment;

FIG. 23 is a flowchart showing operation of the image decoding device inthe second embodiment;

FIG. 24 is a diagram showing a configuration of an image coding devicein a third embodiment;

FIG. 25 is a flowchart (1/2) showing operation of the image codingdevice in the third embodiment;

FIG. 26 is a flowchart (2/2) showing the operation of the image codingdevice in the third embodiment;

FIGS. 27A and 27B are diagrams of assistance in explaining edgedetection using motion compensation;

FIG. 28 is a diagram showing a configuration of an image decoding devicein the third embodiment; and

FIG. 29 is a flowchart showing operation of the image decoding device inthe third embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Modes for carrying out the present technology will hereinafter bedescribed. The present technology prevents a continuous edge fromstraddling a plurality of transform blocks and thus improves subjectiveimage quality by setting transform blocks for an orthogonal transform inconsideration of edge continuity on the basis of a mode of intra-frameprediction (intra prediction). In addition, the present technologyimproves energy concentration by increasing transform blocks notincluding an edge. Further, description will be made of applicability ofthe present technology also to inter-frame prediction (interprediction). Incidentally, description will be made in the followingorder.

-   1. First Embodiment-   1-1. Configuration of Image Coding Device-   1-2. Operation of Image Coding Device-   1-3. Configuration of Image Decoding Device-   1-4. Operation of Image Decoding Device-   2. Second Embodiment-   2-1. Configuration of Image Coding Device-   2-2. Operation of Image Coding Device-   2-3. Configuration of Image Decoding Device-   2-4. Operation of Image Decoding Device-   3. Third Embodiment-   3-1. Configuration of Image Coding Device-   3-2. Operation of Image Coding Device-   3-3. Configuration of Image Decoding Device-   3-4. Operation of Image Decoding Device

1. First Embodiment

[1-1. Configuration of Image Coding Device]

FIG. 1 shows a configuration of an image coding device in a firstembodiment. The image coding device 10 includes an arithmetic section11, a processing selecting switch 12, a first transform section 13, asecond transform section 14, a first quantizing section 15, a secondquantizing section 16, and an entropy coding section 17. The imagecoding device 10 also includes a processing selecting switch 18, a firstdequantizing section 19, a second dequantizing section 20, a firstinverse transform section 21, a second inverse transform section 22, anarithmetic section 23, a reference memory 24, and a predicting section25. The image coding device 10 further includes a reference image edgedetecting section 31, a transform block setting section 32, and a codingcontrol section 40.

The arithmetic section 11 calculates a prediction error of a predictedimage with respect to an input image by subtracting a predicted imagesignal DS18 generated in the predicting section 25 to be described laterfrom an input image signal DS1. The arithmetic section 11 outputs aprediction error signal DS2 indicating the prediction error to theprocessing selecting switch 12.

The processing selecting switch 12 makes switch selection on the basisof transform information DS40 supplied from the coding control section40 to output the prediction error signal DS2 to the first quantizingsection 15 or the second quantizing section 16.

As shown in FIG. 2A, the first transform section 13 includes ahorizontal and vertical DCT section 131. The horizontal and vertical DCTsection 131 performs a horizontal and vertical DCT of the predictionerror signal DS2 supplied from the processing selecting switch 12. Inaddition, the horizontal and vertical DCT section 131 sets transformblocks in which the horizontal and vertical DCT is performed on thebasis of transform block setting information DS32 supplied from thetransform block setting section 32 to be described later. The horizontaland vertical DCT section 131 outputs transform coefficients DS3 obtainedby performing the horizontal and vertical DCT to the first quantizingsection 15.

As shown in FIG. 2B, the first quantizing section 15 has a horizontaland vertical quantizing section 151. The horizontal and verticalquantizing section 151 quantizes the transform coefficients DS3 outputfrom the first transform section 13. The horizontal and verticalquantizing section 151 outputs quantized data DS5 obtained by performingthe quantization to the entropy coding section 17 and the processingselecting switch 18.

The second transform section 14 includes DCT sections provided forrespective prediction directions as oblique directions and a patternselecting switch for selecting a DCT section corresponding to aprediction direction. For example, as will be described later withreference to FIGS. 10A to 10I, suppose that six modes from a predictionmode 3 to a prediction mode 8 whose prediction directions are obliquedirections are provided in an intra-frame prediction mode. In this case,as shown in FIG. 3, the second transform section 14 includes a firstoblique direction pattern DCT section 141 corresponding to the obliquedirection of the prediction mode 3, . . . , and a sixth obliquedirection pattern DCT section 146 corresponding to the oblique directionof the prediction mode 8. The second transform section 14 also includesa pattern selecting switch 140 for selecting a DCT section correspondingto a prediction mode.

The pattern selecting switch 140 supplies the prediction error signalDS2 supplied from the processing selecting switch 12 to one of the firstto sixth oblique direction pattern DCT sections 141 to 146 on the basisof prediction mode information DS20 from the predicting section 25 to bedescribed later. For example, when the prediction mode information DS20indicates the prediction mode 3, the pattern selecting switch 140supplies the prediction error signal DS2 to the first oblique directionpattern DCT section 141 corresponding to the oblique direction of theprediction mode 3. When the prediction mode information DS20 indicatesthe prediction mode 8, the pattern selecting switch 140 supplies theprediction error signal DS2 to the sixth oblique direction pattern DCTsection 146 corresponding to the oblique direction of the predictionmode 8.

The first oblique direction pattern DCT section 141 performs a DCTaccording to a prediction direction on the prediction error signal DS2supplied via the pattern selecting switch 140. In addition, the firstoblique direction pattern DCT section 141 sets transform blocks for theDCT on the basis of the transform block setting information DS32supplied from the transform block setting section 32 to be describedlater. The first oblique direction pattern DCT section 141 outputstransform coefficients DS4 obtained by performing the DCT to the secondquantizing section 16. The second to sixth oblique direction pattern DCTsections 142 to 146 similarly perform a DCT according to a predictiondirection on the prediction error signal DS2, and output resultingtransform coefficients DS4 to the second quantizing section 16. Thus,the second transform section 14 selectively uses the first to sixthoblique direction pattern DCT sections 141 to 146 on the basis of theprediction mode information DS20 to perform a DCT according to aprediction mode in each transform block based on the transform blocksetting information DS32.

The second quantizing section 16 includes quantizing sections providedfor respective prediction directions as oblique directions and a patternselecting switch for selecting a quantizing section corresponding to aprediction direction. For example, suppose that the six modes from theprediction mode 3 to the prediction mode 8 whose prediction directionsare oblique directions are provided. In this case, as shown in FIG. 4,the second quantizing section 16 includes a first oblique directionpattern quantizing section 161 corresponding to the oblique direction ofthe prediction mode 3, . . . , and a sixth oblique direction patternquantizing section 166 corresponding to the oblique direction of theprediction mode 8. The second quantizing section 16 also includes apattern selecting switch 160 for selecting a quantizing sectioncorresponding to a prediction mode.

The pattern selecting switch 160 supplies the transform coefficients DS4supplied from the second transform section 14 to one of the first tosixth oblique direction pattern quantizing sections 161 to 166 on thebasis of the prediction mode information DS20 from the predictingsection 25. For example, when the prediction mode information DS20indicates the prediction mode 3, the pattern selecting switch 160supplies the transform coefficients DS4 obtained in the first obliquedirection pattern DCT section 141 of the second transform section 14 tothe first oblique direction pattern quantizing section 161 correspondingto the prediction mode 3. When the prediction mode information DS20indicates the prediction mode 8, the pattern selecting switch 160supplies the transform coefficients DS4 obtained in the sixth obliquedirection pattern DCT section 146 of the second transform section 14 tothe sixth oblique direction pattern quantizing section 166 correspondingto the oblique direction of the prediction mode 8.

The first oblique direction pattern quantizing section 161 quantizes thetransform coefficients DS4 supplied from the first oblique directionpattern DCT section 141 of the second transform section 14 via thepattern selecting switch 160. In addition, the first oblique directionpattern quantizing section 161 quantizes transform coefficients DS4 ineach transform block on the basis of the transform block settinginformation DS32 supplied from the transform block setting section 32 tobe described later. The first oblique direction pattern quantizingsection 161 outputs quantized data DS6 obtained by performing thequantization to the entropy coding section 17 and the processingselecting switch 18. The second to sixth oblique direction patternquantizing sections 162 to 166 similarly quantize the transformcoefficients DS4 obtained by DCTs according to prediction directions ineach transform block, and output resulting quantized data DS6 to theentropy coding section 17 and the processing selecting switch 18. Thus,the second quantizing section 16 quantizes the transform coefficientsobtained by performing the DCTs according to the prediction directionsin the second transform section 14 in each DCT block in the respectiveprediction directions.

The entropy coding section 17 in FIG. 1 performs entropy coding of thequantized data DS5 supplied from the first quantizing section 15 or thequantized data DS6 supplied from the second quantizing section 16. Theentropy coding section 17 also performs entropy coding of the predictionmode information DS20 generated in the predicting section 25, thetransform information DS40 generated in the coding control section 40,and the like to be described later. The entropy coding section 17outputs coded data DSC obtained by performing the entropy coding.

The processing selecting switch 18 selects an inverse transform methodon the basis of the transform information DS40 supplied from the codingcontrol section 40. The processing selecting switch 18 outputs thequantized data DS5 from the first quantizing section 15 to the firstdequantizing section 19, and outputs the quantized data DS6 from thesecond quantizing section 16 to the second dequantizing section 20.

As shown in FIG. 5A, the first dequantizing section 19 has a horizontaland vertical dequantizing section 191. The horizontal and verticaldequantizing section 191 dequantizes the quantized data DS5 supplied viathe processing selecting switch 18. In addition, the horizontal andvertical dequantizing section 191 dequantizes the quantized data in eachtransform block corresponding to that of the first quantizing section 15on the basis of the transform block setting information DS32 suppliedfrom the transform block setting section 32. The first dequantizingsection 19 outputs transform coefficients DS11 obtained by performingthe dequantization to the first inverse transform section 21.

As shown in FIG. 5B, the first inverse transform section 21 has ahorizontal and vertical inverse DCT section 211. The horizontal andvertical inverse DCT section 211 subjects the transform coefficientsDS11 supplied from the first dequantizing section 19 to an inverse DCTin the horizontal and vertical directions which inverse DCT correspondsto the DCT in the horizontal and vertical directions in the firsttransform section 13. The horizontal and vertical inverse DCT section211 outputs a prediction error signal DS13 obtained by performing theinverse DCT to the arithmetic section 23.

The second dequantizing section 20 is configured to performdequantization corresponding to the quantization performed in the secondquantizing section 16. For example, as shown in FIG. 6, the seconddequantizing section 20 includes a pattern selecting switch 200 and afirst oblique direction pattern dequantizing section 201 to a sixthoblique direction pattern dequantizing section 206.

The pattern selecting switch 200 supplies the quantized data DS6supplied via the processing selecting switch 18 to one of the first tosixth oblique direction pattern dequantizing sections 201 to 206 on thebasis of the prediction mode information DS20 from the predictingsection 25. For example, when the prediction mode information DS20indicates the prediction mode 3, the pattern selecting switch 200supplies the quantized data DS6 obtained in the first oblique directionpattern quantizing section 161 of the second quantizing section 16 tothe first oblique direction pattern dequantizing section 201corresponding to the prediction mode 3. Similarly, when the predictionmode information DS20 indicates the prediction mode 8, the patternselecting switch 200 supplies the quantized data DS6 obtained in thesixth oblique direction pattern quantizing section 166 of the secondquantizing section 16 to the sixth oblique direction patterndequantizing section 206 corresponding to the prediction mode 8.

The first oblique direction pattern dequantizing section 201 subjectsthe quantized data DS6 supplied via the pattern selecting switch 200 todequantization corresponding to the quantization of the first obliquedirection pattern quantizing section 161 in the second quantizingsection 16. In addition, the first oblique direction patterndequantizing section 201 dequantizes the quantized data in eachtransform block corresponding to that of the second quantizing section16 on the basis of the transform block setting information DS32 suppliedfrom the transform block setting section 32. The first oblique directionpattern dequantizing section 201 outputs transform coefficients DS12obtained by performing the dequantization to the second inversetransform section 22. In addition, the second to sixth oblique directionpattern dequantizing sections 202 to 206 similarly dequantize thesupplied quantized data DS6, and output resulting transform coefficientsDS12 to the second inverse transform section 22. Thus, the seconddequantizing section 20 performs dequantization in correspondence withthe quantization of the second quantizing section 16.

The second inverse transform section 22 is configured to perform aninverse DCT corresponding to the DCT performed in the second transformsection 14. For example, as shown in FIG. 7, the second inversetransform section 22 includes a pattern selecting switch 220 and a firstoblique direction pattern inverse DCT section 221 to a sixth obliquedirection pattern inverse DCT section 226.

The pattern selecting switch 220 supplies the transform coefficientsDS12 supplied from the second dequantizing section 20 to one of thefirst to sixth oblique direction pattern inverse DCT sections 221 to 226on the basis of the prediction mode information DS20 from the predictingsection 25. For example, when the prediction mode information DS20indicates the prediction mode 3, the pattern selecting switch 220supplies the transform coefficients DS12 obtained in the first obliquedirection pattern dequantizing section 201 in the second dequantizingsection 20 to the first oblique direction pattern inverse DCT section221 corresponding to the prediction mode 3. Similarly, when theprediction mode information DS20 indicates the prediction mode 8, thepattern selecting switch 220 supplies the transform coefficients DS12obtained in the sixth oblique direction pattern dequantizing section 206in the second dequantizing section 20 to the sixth oblique directionpattern inverse DCT section 226 corresponding to the prediction mode 8.

The first oblique direction pattern inverse DCT section 221 subjects thetransform coefficients DS12 supplied via the pattern selecting switch220 to an inverse DCT corresponding to the DCT of the first obliquedirection pattern DCT section 141 in the second transform section 14.The first oblique direction pattern inverse DCT section 221 performs aninverse DCT of transform coefficients in each transform blockcorresponding to that of the second transform section 14 on the basis ofthe transform block setting information DS32 supplied from the transformblock setting section 32. The first oblique direction pattern inverseDCT section 221 outputs a prediction error signal DS14 obtained byperforming the inverse DCT to the arithmetic section 23. In addition,the second to sixth oblique direction pattern inverse DCT sections 222to 226 similarly perform an inverse DCT of the supplied transformcoefficients DS12, and output a resulting prediction error signal DS14to the arithmetic section 23. Thus, the second inverse transform section22 performs the inverse DCTs corresponding to the DCTs according to theprediction directions in the second transform section 14.

The arithmetic section 23 generates a reference image signal DS15 byadding the predicted image signal DS18 generated in the predictingsection 25 to the prediction error signal DS13 supplied from the firstinverse transform section 21 or the prediction error signal DS14supplied from the second inverse transform section 22. The arithmeticsection 23 stores the generated reference image signal DS15 in thereference memory 24.

The reference image signal DS15 stored in the reference memory 24 issupplied to the predicting section 25 and the reference image edgedetecting section 31.

The predicting section 25 performs intra-frame prediction in eachprediction mode using the reference image signal DS15. In addition, thepredicting section 25 determines a prediction mode that maximizes codingefficiency, and generates prediction mode information DS20 indicatingthe prediction mode that maximizes coding efficiency. The predictingsection 25 outputs the generated prediction mode information DS20 to thesecond transform section 14, the second quantizing section 16, theentropy coding section 17, the second dequantizing section 20, thesecond inverse transform section 22, and the transform block settingsection 32. Further, the predicting section 25 generates the predictedimage signal DS18 in the prediction mode that maximizes codingefficiency, and outputs the predicted image signal DS18 to thearithmetic sections 11 and 23.

The reference image edge detecting section 31 detects an edge using animage signal of a coded adjacent bock stored in the reference memory 24,and outputs an index DS31 indicating the position of the edge and theintensity (steepness of change in density) of the edge to the transformblock setting section 32.

The transform block setting section 32 estimates the continuity of theedge within a sub-block as a coding object on the basis of the indexDS31 supplied from the reference image edge detecting section 31 and theprediction mode information DS20 supplied from the predicting section25. The transform block setting section 32 divides the sub-block as thecoding object from a result of the estimation, sets transform blocks inan orthogonal transform and quantization, and generates transform blocksetting information DS32 indicating the set transform blocks. Thetransform block setting section 32 outputs the generated transform blocksetting information DS32 to the first transform section 13, the secondtransform section 14, the first quantizing section 15, the secondquantizing section 16, the first dequantizing section 19, the seconddequantizing section 20, the first inverse transform section 21, and thesecond inverse transform section 22.

The coding control section 40 generates transform information DS40. Thetransform information DS40 is information for selecting either a processof performing a horizontal and vertical DCT in relation to theorthogonal transform and horizontal and vertical quantization or aprocess of performing a one-dimensional DCT and quantization along theprediction direction indicated by the prediction mode information DS20.The coding control section 40 outputs the generated transforminformation DS40 to the processing selecting switch 12, the entropycoding section 17, and the processing selecting switch 18.

[1-2. Operation of Image Coding Device]

The operation of the image coding device will next be described. In thecase of a luminance signal in the intra-frame prediction of theH.264/AVC system, for example, a plurality of macroblocks are set in acoding object frame as shown in FIG. 8A. FIG. 8B shows a macroblockhaving 16 sub-blocks of 4×4 pixels. FIG. 8C shows a macroblock havingfour sub-blocks of 8×8 pixels. FIG. 8D shows a macroblock having onesub-block of 16×16 pixels.

In the H.264/AVC system, four modes, that is, prediction modes 0 to 3are set as prediction modes for sub-blocks of 16×16 pixels. In addition,nine prediction modes, that is, prediction modes 0 to 8 are set asprediction modes for sub-blocks of 8×8 pixels. Further, nine predictionmodes, that is, prediction modes 0 to 8 are set as prediction modes forsub-blocks of 4×4 pixels.

FIG. 9 is a diagram of assistance in explaining positional relationbetween for example pixels a to p belonging to a sub-block of 4×4 pixelsand pixels A to M adjacent to the sub-block in adjoining blocks on aleft side, an upper left side, an upper side, and an upper right side ofthe sub-block.

FIGS. 10A to 10I represent prediction modes for 4×4 pixels inintra-frame prediction. Incidentally, arrows in FIGS. 10A to 10Iindicate a prediction direction. FIG. 10A represents a prediction mode 0(vertical). The prediction mode 0 generates predicted values from thereference pixels A to D adjoining in a vertical direction. FIG. 10Brepresents a prediction mode 1 (horizontal). As indicated by arrows, theprediction mode 1 generates predicted values from the reference pixels Ito L adjoining in a horizontal direction. FIG. 10C represents aprediction mode 2 (DC). The prediction mode 2 generates predicted valuesfrom the reference pixels A to D and I to L adjoining in the verticaldirection and the horizontal direction of the block among the 13reference pixels A to M.

FIG. 10D represents a prediction mode 3 (diagonal down-left). Theprediction mode 3 generates predicted values from the reference pixels Ato H continuous in the horizontal direction among the 13 referencepixels A to M. FIG. 10E represents a prediction mode 4 (diagonaldown-right). The prediction mode 4 generates predicted values from thereference pixels A to D and I to M adjacent to the block in questionamong the 13 reference pixels A to M. FIG. 10F represents a predictionmode 5 (vertical-right). The prediction mode 5 generates predictedvalues from the reference pixels A to D and I to M adjacent to the blockin question among the 13 reference pixels A to M.

FIG. 10G represents a prediction mode 6 (horizontal-down). As with theprediction mode 4 and the prediction mode 5, the prediction mode 6generates predicted values from the reference pixels A to D and I to Madjacent to the block in question among the 13 reference pixels A to M.FIG. 10H represents a prediction mode 7 (vertical-left). The predictionmode 7 generates predicted values from the four reference pixels A to Dadjoining on the upper side of the block in question and the threereference pixels E to G following the four reference pixels A to D amongthe 13 reference pixels A to M. FIG. 10I represents a prediction mode 8(horizontal-up). The prediction mode 8 generates predicted values fromthe four reference pixels I to L adjoining on the left side of the blockin question among the 13 reference pixels A to M.

The predicting section 25 generates a predicted image signal DS18 ineach of the above-described prediction modes. In addition, the codingcontrol section 40 generates the transform information DS40 according toa prediction mode selected by the predicting section 25. For example,the coding control section 40 generates the transform information DS40as information for selecting either a process of performing a horizontaland vertical DCT in relation to an orthogonal transform and horizontaland vertical quantization or a process of performing a one-dimensionalDCT and quantization along the prediction direction indicated by theprediction mode information DS20.

When the transform information DS40 indicates a prediction mode of thehorizontal and vertical directions, the processing selecting switch 12supplies the prediction error signal DS2 to the first transform section13 so that a DCT in the horizontal and vertical directions is performed.In addition, when the transform information DS40 indicates an obliquedirection prediction mode, the processing selecting switch 12 suppliesthe prediction error signal DS2 to the second transform section 14 sothat a one-dimensional DCT is performed along the prediction direction.

When the transform information DS40 indicates a prediction mode of thehorizontal and vertical directions, the processing selecting switch 18supplies the quantized data DS5 to the first dequantizing section 19 sothat the quantized data obtained by a DCT and quantization in thehorizontal and vertical directions is subjected to correspondingdequantization and a corresponding inverse transform. When the transforminformation DS40 indicates an oblique direction prediction mode, theprocessing selecting switch 18 supplies the quantized data DS6 to thesecond dequantizing section 20 so that the quantized data obtained by aone-dimensional DCT and quantization in the oblique direction issubjected to corresponding dequantization and a corresponding inversetransform.

Description in the following will be made of determination of aprediction mode that maximizes coding efficiency. The predicting section25 performs a coding process in each of the prediction modes, anddetermines that a prediction mode that minimizes coding cost obtained asa result of the coding process is an optimum mode. Specifically, acoding cost K is calculated by using Equation (1), and a prediction modethat minimizes the coding cost K is set as an optimum mode.K=SAD+λ×OH  (1)where a difference error SAD is an absolute value of a difference valuebetween the predicted image signal generated by a prediction methoddefined in the prediction mode and the input image signal, sideinformation OH is an amount of various information necessary when theprediction mode is used, and a coefficient λ is a Lagrange multiplier.

In addition, the determination of the optimum mode is not limited to thecase of using the side information and the absolute value of thedifference value, but the mode may be determined using only the modeinformation or only an absolute sum of a prediction error signal, orvalues obtained by performing a Hadamard transform or approximation ofthese pieces of information may be used. In addition, the coding cost Kmay be obtained by using the activity of the input image, or the codingcost may be obtained using a quantization scale.

The coding cost K can also be calculated by using Equation (2).K=D+λ×R  (2)where a coding distortion D represents a square error between the inputimage signal and a local decoded image signal, an amount of code R isestimated by tentative coding, and a coefficient λ is a constantdetermined on the basis of a quantization parameter.

When the coding cost is calculated by using Equation (2), the imagecoding device 10 needs entropy coding and local decoding (includingdequantization and inverse transform processing) for each mode. Thus,though a circuit scale is increased, an accurate amount of code andaccurate coding distortion can be used, and the coding efficiency can bemaintained at a high level.

Description will next be made of a DCT performed in the H.264/AVC systemas a method of a one-dimensional DCT along a prediction direction.Letting X be the input image signal and letting T be a transform matrixfor a sub-block of 4×4 pixels, a transform coefficient C is obtainedaccording to Equation (3).

$\begin{matrix}{{C = {T \cdot X \cdot T^{t}}}{{{where}\mspace{14mu} T} = \begin{bmatrix}1 & 1 & 1 & 1 \\2 & 1 & {- 1} & {- 2} \\1 & {- 1} & {- 1} & 1 \\1 & {- 2} & 2 & {- 1}\end{bmatrix}}} & (3)\end{matrix}$

That is, the input image signal is subjected to a one-dimensional DCT inthe horizontal direction using the transform matrix T, and thereaftersubjected to a one-dimensional DCT in the vertical direction using thetransposed matrix T^(t) of the transform matrix T. This operation alsoapplies to a sub-block of 8×8 pixels.

Next, a DCT along a prediction direction is performed for the predictionmodes in the six directions excluding the three types of the predictionmode 0 (vertical), the prediction mode 1 (horizontal), and theprediction mode 2 (DC) among the nine types of prediction modes ofintra-frame prediction defined in the H.264/AVC system.

Description in the following will be made of a case in which a DCT alonga prediction direction is performed for the sub-block of 4×4 pixelsshown in FIG. 9. In the case of the prediction mode 3, the pixels withinthe sub-block are grouped into seven sets of (a), (b, e), (c, f, i), (d,g, j, m), (h, k, n), (l, o), and (p) along the prediction direction.

FIG. 11A shows a combination of pixel strings in which a one-dimensionalDCT is performed. Letting F(u) be a transform coefficient, aone-dimensional DCT with a base length of four is performed for thepixel string (d, g, j, m) present in a diagonal line of the 4×4 pixelblock.

Equation (4) represents the transform coefficient F(u) when the baselength is “N.”

$\begin{matrix}{{F(u)} = {\sqrt{\frac{2}{N}}{C(u)}{\sum\limits_{x = 0}^{N - 1}{{f(x)}\cos\left\{ \frac{{\pi\left( {{2\; x} + 1} \right)}u}{2N} \right\}}}}} & (4)\end{matrix}$

When the transform coefficient F(u) is obtained for the pixel string (d,g, j, m), the transform coefficient F(u) can be obtained by performingthe operation of Equation (4) with the base length “N=4” and with“f(0)=d, f(1)=g, f(2)=j, and f(3)=m.”

When the transform coefficient F(u) is obtained for the pixel string (c,f, i), the transform coefficient F(u) can be obtained by performing theoperation of Equation (4) with the base length “N=3” and with “f(0)=c,f(1)=f, and f(2)=i.”

For the pixel (a) and the pixel string (b, e), the transform coefficientF(u) is obtained with the pixels combined with each other in afolded-back manner. In this case, the transform coefficient F(u) can beobtained by performing the operation of Equation (4) with the baselength “N=3” and with “f(0)=b, f(1)=e, and f(2)=a” or “f(0)=e, f(1)=b,and f(2)=a.”

FIG. 11B represents a case in which a one-dimensional DCT is performedwithout pixels combined with each other in a folded-back manner. Aone-dimensional DCT similar to that of FIG. 11A is performed for thepixel strings (d, g, j, m) and (c, f, i). A one-dimensional DCT isperformed for each of the pixel string (b, e) and the pixel (a) withoutthe pixels being combined with each other in a folded-back manner.Specifically, when the transform coefficient F(u) is obtained for thepixel string (b, e), the transform coefficient F(u) can be obtained byperforming the operation of Equation (4) with the base length “N=2” andwith “f(0)=b and f(1)=e.” When the transform coefficient F(u) isobtained for the pixel (a), the transform coefficient F(u) can beobtained by performing the operation of Equation (4) with the baselength “N=1” and with “f(0)=a.”

FIG. 11C represents a case in which a one-dimensional DCT is performedwith all of the pixel strings (c, f, i) and (e, b) and the pixel (a)combined with each other in a folded-back manner. A one-dimensional DCTsimilar to that of FIG. 11A is performed for the pixel string (d, g, j,m). For the pixel strings (c, f, i) and (e, b) and the pixel (a), aone-dimensional DCT is performed with the pixels combined with eachother in a folded-back manner. Specifically, when the transformcoefficient F(u) is obtained for the pixel string (c, f, i, e, b, a),the transform coefficient F(u) can be obtained by performing theoperation of Equation (4) with the base length “N=6” and with “f(0)=c,f(1)=f, f(2)=i, f(3)=e, f(4)=b, and f(5)=a” or “f(0)=i, f(1)=f, f(2)=c,f(3)=b, f(4)=e, and f(5)=a.” Incidentally, the transform coefficientF(u) can be obtained in a similar manner for the pixel strings (h, k, n)and (l, o) and the pixel (p).

In the case of the prediction mode 4, it can be considered that theprediction direction of the prediction mode 3 is horizontally reversedin the prediction mode 4, and therefore DCT operation can be performedin a similar manner to that of the prediction mode 3.

Description will next be made of a DCT operation method corresponding tothe prediction modes 5 to 8. In the case of the prediction mode 5, thepixels within the sub-block are grouped into five sets of pixel strings(a, e, j, n), (b, f, k, o), (c, g, l, p), (i, m), and (d, h) along theprediction direction. FIG. 12A shows the combination of the pixelstrings to which a one-dimensional DCT is applied. When the transformcoefficient F(u) is obtained for the pixel string (a, e, j, n), thetransform coefficient F(u) can be obtained by performing the operationof Equation (4) with the base length “N=4” and with “f(0)=a, f(1)=e,f(2)=j, and f(3)=n.” For the pixel strings (b, f, k, o) and (c, g, l,p), the transform coefficient F(u) can be obtained by performing similaroperation.

When the transform coefficient F(u) is obtained for the pixel string (i,m), the transform coefficient F(u) can be obtained by performing theoperation of Equation (4) with the base length “N=2” and with “f(0)=iand f(1)=m.” For the pixel string (d, h), the transform coefficient F(u)can be obtained by performing similar operation.

In addition, a method of performing a one-dimensional DCT with pixelscombined with each other in a folded-back manner can be considered asanother example of DCT operation corresponding to the prediction mode 5.FIG. 12B shows a combination of pixel strings to which a one-dimensionalDCT is applied in this case. A one-dimensional DCT similar to that ofthe above-described one-dimensional DCT is performed for the pixelstring (b, f, k, o). For the pixel strings (a, e, j, n) and (m, i) andthe pixel strings (p, l, g, c) and (d, h), a one-dimensional DCT isperformed with the pixels combined with each other in a folded-backmanner. Specifically, when the transform coefficient F(u) is obtainedfor the pixel string (a, e, j, n, m, i), the transform coefficient F(u)can be obtained by performing the operation of Equation (4) with thebase length “N=6” and with “f(0)=a, f(1)=e, f(2)=j, f(3)=n, f(4)=m, andf(5)=i.” In addition, the transform coefficient F(u) can be obtained byperforming similar operation for the pixel strings (p, l, g, c) and (d,h).

In the case of the prediction modes 6 to 8, it can be considered thatthe prediction direction of the prediction mode 5 is rotated or reversedin the prediction modes 6 to 8, and therefore DCT operation can beperformed in a similar manner to that of the prediction mode 5.

In addition, as for the block size of sub-blocks as coding objects, thecoding process can also be performed with blocks of smaller sizes thanthe 4×4 pixel size or blocks of larger sizes than the 4×4 pixel size assub-blocks. In addition, the DCT operation is not limited to real-numberDCT operation, but the DCT can also be performed by integer operation.

FIG. 13 and FIG. 14 are flowcharts of operation of the image codingdevice 10 in the first embodiment. Incidentally, suppose that theprediction modes 0 to 8 shown in FIGS. 10A to 10I are provided in thecoding process.

In step ST1, the image coding device 10 obtains an input image. Theimage coding device 10 obtains an input image signal DS1, and startscoding in each macroblock or each macroblock pair.

In step ST2, the image coding device 10 performs initialization relatingto sub-blocks. The image coding device 10 initializes a sub-block indexsub_blk to “sub_blk=0” and sets a maximum sub-block number MAX_SUB_BLKat the same time. The image coding device 10 then proceeds to step ST3.

In step ST3, the image coding device 10 determines whether the sub-blockindex sub_blk is smaller than the maximum sub-block number MAX_SUB_BLK.When the sub-block index sub_blk is smaller than the maximum sub-blocknumber MAX_SUB_BLK, there is a sub-block yet to be coded amongsub-blocks within a macroblock, and therefore the image coding device 10proceeds to step ST4. When the sub-block index sub_blk is not smallerthan the maximum sub-block number MAX_SUB_BLK, there is no sub-block yetto be coded among the sub-blocks within the macroblock, and thereforethe image coding device 10 proceeds to step ST22.

In step ST4, the image coding device 10 performs initialization relatingto prediction modes. The image coding device 10 initializes a predictionmode index mode_idx to “mode_idx=0,” and sets a maximum selectable modenumber MAX_MODE. For example, when nine prediction modes, or theprediction modes 0 to 8, are provided, “MAX_MODE=9.” Incidentally, aprediction mode index mode_idx=0 corresponds to the prediction mode 0.Similarly, the indexes mode_idx=1 to 8 correspond to the predictionmodes 1 to 8.

In step ST5, the image coding device 10 determines whether theprediction mode index mode_idx is smaller than the maximum mode numberMAX_MODE. When the prediction mode index mode_idx is smaller than themaximum mode number MAX_MODE, not all intra-frame prediction modes havebeen tried, and therefore the image coding device 10 proceeds to stepST6. When the prediction mode index mode_idx is not smaller than themaximum mode number MAX_MODE, all the intra-frame prediction modes havebeen tried, and therefore the image coding device 10 proceeds to stepST21.

In step ST6, the image coding device 10 sets transform informationtrans_idx. The image coding device 10 sets the transform informationtrans_idx according to the value of the prediction mode index mode_idx.When the prediction mode index mode_idx indicates an oblique directionprediction mode (prediction mode 3 to 8), the image coding device 10sets the transform information trans_idx to “trans_idx=0.” The imagecoding device 10 then proceeds to step ST7. When the prediction modeindex mode_idx indicates a non-oblique direction prediction mode(prediction mode 0 to 2), the image coding device 10 sets the transforminformation trans_idx to “trans_idx=1.” The image coding device 10 thenproceeds to step ST7.

In step ST7, the image coding device 10 generates a predicted imagesignal in the prediction mode of the index mode_idx. The image codingdevice 10 generates the predicted image signal in the prediction modeindicated by the index mode_idx using the image signal of a referenceimage. The image coding device 10 then proceeds to step ST8.

In step ST8, the image coding device 10 generates a prediction errorsignal. The image coding device 10 generates the prediction error signalDS2 by calculating difference between the predicted image signal DS18 inthe generated predicted image signal DS18 in the prediction mode of theindex mode_idx and the input image signal DS1. The image coding device10 then proceeds to step ST9.

In step ST9, the image coding device 10 performs edge detection. Theimage coding device 10 detects an edge using the image signal of thestored reference image (image signal of a coded adjacent block), andgenerates an index DS31 indicating the position of the edge and theintensity of the edge. The image coding device 10 then proceeds to stepST10.

In step ST10, the image coding device 10 sets transform blocks. Theimage coding device 10 estimates the continuity of the edge within thesub-block as a coding object on the basis of the index DS31 indicatingthe position of the edge and the intensity of the edge and the directionof the prediction mode indicated by the index mode_idx. Further, theimage coding device 10 sets transform blocks on the basis of a result ofthe estimation, and generates transform block setting information DS32.The image coding device 10 then proceeds to step ST11.

In step ST11, the image coding device 10 determines whether theprediction mode index mode_idx is smaller than a minimum mode numbermode_direction of an oblique direction prediction mode as a minimumvalue of the mode numbers of the oblique direction prediction modes, orwhether the transform information is “trans_idx=1.” The image codingdevice 10 proceeds to step ST12 in at least one of the cases where theprediction mode index mode_idx is smaller than the minimum mode numbermode_direction and where the transform information is “trans_idx=1.”Otherwise, the image coding device 10 proceeds to step ST14.

In step ST12, the image coding device 10 performs a horizontal andvertical DCT. The image coding device 10 then proceeds to step ST13. Instep ST13, the image coding device 10 performs horizontal and verticalquantization. The image coding device 10 then proceeds to step ST16. Theimage coding device 10 for example changes the processing selectingswitch 12 to the side of the first transform section 13, and performsthe DCT and the quantization using the first transform section 13 andthe first quantizing section 15.

In step ST14, the image coding device 10 performs an oblique directionpattern DCT. The image coding device 10 then proceeds to step ST15. Instep ST15, the image coding device 10 performs oblique direction patternquantization. The image coding device 10 then proceeds to step ST16. Theimage coding device 10 for example changes the processing selectingswitch 12 to the side of the second transform section 14. In addition,the image coding device 10 changes the pattern selecting switch 140 inthe second transform section 14 and the pattern selecting switch 160 inthe second quantizing section 16 according to the prediction mode indexmode_idx. The image coding device 10 performs the DCT and thequantization using an oblique direction pattern DCT section and anoblique direction pattern quantizing section corresponding to theprediction direction by changing the switches according to the indexmode_idx.

In step ST16, the image coding device 10 performs entropy coding. Theimage coding device 10 entropy-codes the quantized data DS5 and DS6, theprediction mode information DS20, and the transform information DS40.The image coding device 10 then proceeds to step ST17.

In step ST17, the image coding device 10 stores the coding cost of theprediction mode. The image coding device 10 calculates a cost value K asdescribed above, and stores the calculated cost value K. The imagecoding device 10 then proceeds to step ST18.

In step ST18, the image coding device 10 determines whether thetransform information trans_idx is “trans_idx=0.” When the transforminformation trans_idx is “trans_idx=0,” the image coding device 10proceeds to step ST19. When the transform information trans_idx is not“trans_idx=0,” the image coding device 10 proceeds to step ST20.

In step ST19, the image coding device 10 adds “1” to the transforminformation trans_idx to set new transform information trans_idx. Theimage coding device 10 then returns to step ST11.

In step ST20, the image coding device 10 adds “1” to the prediction modeindex mode_idx to set a new index mode_idx. The image coding device 10then returns to step ST5.

By similarly repeating the process from step ST5, coding cost iscalculated in all possible prediction modes for the sub-block.

Thereafter, when the image coding device 10 determines in step ST5 thatthe prediction mode index mode_idx is not smaller than the maximum modenumber MAX_MODE, the image coding device 10 proceeds to step ST21. Instep ST21, the image coding device 10 adds “1” to the sub-block indexsub_blk to set a new index sub_blk. The image coding device 10 thenreturns to step ST3.

When the image coding device 10 determines in step ST3 that thesub-block index sub_blk is not smaller than the maximum sub-block numberMAX_SUB_BLK, and therefore proceeds to step ST22, the image codingdevice 10 loads data on an optimum mode for each sub-block. The imagecoding device 10 compares the coding costs of the respective modes whichcoding costs are obtained for each sub-block with each other, and loadsdata on an optimum mode in each sub-block. The image coding device 10then proceeds to step ST23. In addition, the image coding device 10generates the prediction mode information DS20 indicating the optimummode.

In step ST23, the image coding device 10 multiplexes and sends out codeddata obtained by performing coding in the optimum mode in themacroblock. In addition, the image coding device 10 entropy-codes theprediction mode information DS20, that is, the index mode_idx of theoptimum mode and the transform information DS40, that is, the transforminformation trans_idx, and includes the entropy-coded information in thecoded data.

The operation of the reference image edge detecting section 31 will nextbe described with reference to FIGS. 15A to 15F. FIG. 15A represents acase where the size of a sub-block as a coding object is 16×16 pixels.Parts indicated by hatching in FIG. 15A are the signals of one verticalline and one horizontal line of coded adjacent blocks stored in thereference memory 24. The reference image edge detecting section 31detects an edge of the signal of each of the one adjacent vertical lineand the one adjacent horizontal line represented by the hatched parts,and outputs an index indicating the position of the edge and theintensity of the edge. For example, the reference image edge detectingsection 31 applies a one-dimensional Sobel filter, and outputs a signalobtained as a result of applying the one-dimensional Sobel filter. Thesignal obtained as a result of applying the one-dimensional Sobel filterhas information on both the position of the edge and the intensity ofthe edge. An image of the signal obtained as a result of applying theone-dimensional Sobel filter is shown in FIG. 15B. Incidentally, for thesimplicity of description, the signal as a multivalued signal is shownin white and black for convenience, and positions shown in blackrepresent an edge. The format of the index indicating the position ofthe edge and the intensity of the edge may not be that of the signalobtained as a result of applying the filter. For example, a flagindicating the presence or absence of an edge and data in the form ofpoints indicating the intensity of the edge only at the position of theedge may be obtained. In addition, any method may be used to calculatethe index indicating the position of the edge and the intensity of theedge as long as the index can be obtained. For example, a Prewitt filteror a Laplacian filter may be used, or a mathematical or physical methodother than filters may be used.

In addition, while the index indicating the position of the edge and theintensity of the edge of the signal of each of the one vertical line andthe one horizontal line adjacent to the sub-block as a coding object isobtained in FIGS. 15A and 15B, there is no limitation to this. As shownin FIGS. 15C and 15D, a plurality of lines may be used. Further, asimilar process can be applied even when the size of the sub-block isdifferent from the size of 16×16 pixels. For example, as shown in FIGS.15E and 15F, a similar process can be applied also to a case of 8×8pixels.

The operation of the transform block setting section 32 will next bedescribed with reference to FIGS. 16A to 16D. FIG. 16A represents a casewhere the block size of a sub-block as a coding object is 16×16 pixels.The transform block setting section 32 divides the sub-block on thebasis of information indicating the position of an edge and theintensity of the edge and the prediction mode information DS20, and setstransform blocks. The transform block setting section 32 estimates thecontinuity of the edge within the sub-block as the coding object on thebasis of the position of the edge and the prediction mode informationDS20. For example, the transform block setting section 32 estimates thatedges detected by applying a Sobel filter to the vertical and horizontallines of adjacent blocks are continuous in a prediction directionindicated by the prediction mode information DS20 as shown in FIG. 16A.Incidentally, FIG. 16A shows a result of estimation of edge continuitywhen the prediction mode information DS20 indicates the prediction mode5. In addition, in the figure, for the simplicity of description, amultivalued signal is shown in white and black for convenience, andblack parts represent an edge.

In addition, as shown in FIG. 16B, when the direction of edges can beestimated with the positions and intensities of the edges detected in aplurality of lines in adjacent blocks, it is estimated that the edgesare present in directions considered to be more appropriate from theprediction mode and the directions of the edges in the adjacent blocks.For example, the reliability of the prediction mode and the reliabilityof the directions of the edges in the adjacent blocks are converted intonumerical values, and the higher of the numerical values is employed.The conversion of reliability into a numerical value includes a methodof making the reliability of an edge in an adjacent block higher whenthe edge has a high intensity. When the numerical value exceeds athreshold value, the direction of the edge in the adjacent block isemployed, for example. FIG. 16B shows an example in which the directionsof the edges in the adjacent blocks, rather than the prediction mode 5,are employed.

Next, in consideration of the estimated continuity of the edges, thetransform block setting section 32 divides the sub-block as a codingobject, and sets transform blocks for an orthogonal transform. In thiscase, the sub-block as a coding object is desirably divided into aplurality of transform blocks such that areas not covering an edge areas large as possible from a viewpoint of coding efficiency. Thus, asshown in FIG. 16C, the transform block setting section 32 divides thesub-block as a coding object horizontally and vertically such thatboundaries between blocks after the division do not include an edge. Forexample, the transform block setting section 32 finds a terminal part ofa continuous edge, and determines a boundary of division in such amanner as to be in contact with the terminal part. The transform blocksetting section 32 sets blocks after the division as transform blocks asunits of an orthogonal transform, and generates the transform blocksetting information DS32 indicating the transform blocks.

In dividing a sub-block, there may be a case in which a plurality ofedges are detected and the sub-block cannot be divided such thatboundaries of transform blocks do not cover an edge. In that case, thetransform block setting section 32 determines the order of priority ofedges not to fall on a block boundary from indexes indicating theintensity of the edges. For example, the transform block setting section32 gives higher priority to a higher-energy signal obtained by thefiltering of a Sobel filter. The order of priority may be determined ina relative manner or an absolute manner. The order of priority is thusdetermined, and division is made such that an edge of high priority doesnot fall on the boundaries between the transform blocks. In addition,sub-blocks do not necessarily need to be divided in both the horizontaland vertical directions, but may be divided in one of the directions, ormay not be divided at all. The blocks after the division may be furtherdivided hierarchically, or division into a plurality of blocks may bemade in a same layer. Further, the size of the transform blocks afterthe division may be limited to sizes of two raised to nth power inconsideration of compatibility with ordinary coding systems.

Further, when a signal is output in a different format from thereference image edge detecting section 31, it suffices for the transformblock setting section 32 to estimate the continuity of an edge on thebasis of a concept similar to that of the above-described procedureaccording to the signal, and set transform blocks.

The size of sub-blocks is not limited to the 16×16 pixel size, but asimilar process can be applied to other sub-block sizes. For example, asshown in FIG. 16D, a similar process can be applied to a sub-block sizeof 8×8 pixels.

FIG. 17 represents a procedure for setting transform blocks using thereference image edge detecting section 31 and the transform blocksetting section 32. In step ST31, the reference image edge detectingsection 31 obtains an index indicating the position of an edge in areference image adjacent to a sub-block as a coding object and theintensity of the edge. The process then proceeds to step ST32.

In step ST32, the transform block setting section 32 estimates thecontinuity of the edge within the sub-block as the coding object on thebasis of the index indicating the position of the edge and the intensityof the edge and the prediction mode information DS20 obtained from thepredicting section 25. The process then proceeds to step ST33.

In step ST33, the transform block setting section 32 sets transformblocks in consideration of the estimated continuity of the edge and theintensity of the edge.

After the transform blocks are thus set in the transform block settingsection 32, the first transform section 13, the first quantizing section15, the first dequantizing section 19, and the first inverse transformsection 21 perform a DCT, quantization, dequantization, and an inverseDCT in each of the set transform blocks.

In addition, the second transform section 14, the second quantizingsection 16, the second dequantizing section 20, and the second inversetransform section 22 perform a DCT, quantization, dequantization, and aninverse DCT in the prediction direction corresponding to the predictionmode information DS20 in each of the set transform blocks. FIGS. 18A to18E represent a case in which a sub-block has a block size of 8×8pixels, and the sub-block is divided into four parts, which are set astransform blocks, as shown in FIG. 16D. For example, the sub-block of8×8 pixels shown in FIG. 18A is divided into transform blocks of 5×6pixels, 3×6 pixels, 5×2 pixels, and 3×2 pixels, as shown in FIGS. 18B to18E. In addition, suppose that the prediction mode is the mode 5. Inthis case, signals within the blocks are scanned and DCTs are performedin consideration of correlation present in a vertical oblique direction.For example, scans as indicated by arrows in FIGS. 18B to 18E areperformed, and DCTs are performed according to the respective numbers ofcontinuous pixels.

Incidentally, the orthogonal transform is not limited to DCTs, but mayfor example be wavelet transforms, Hadamard transforms, or transformsobtained by reducing the wavelet transforms and the Hadamard transformsto integer precision. It suffices to use a method appropriate for anorthogonal transform also employing quantization.

[1-3. Configuration of Image Decoding Device]

Description will next be made of an image decoding device that decodesthe coded data generated by the image coding device 10.

FIG. 19 shows a configuration of the image decoding device 50 in thefirst embodiment. The image decoding device 50 includes an entropydecoding section 51, a processing selecting switch 52, a firstdequantizing section 53, a second dequantizing section 54, a firstinverse transform section 55, a second inverse transform section 56, anarithmetic section 57, a reference memory 58, and a predicting section60. The image decoding device 50 also includes a reference image edgedetecting section 71, a transform block setting section 72, and adecoding control section 80.

The entropy decoding section 51 entropy-decodes the coded data DSCreceived as input. The entropy decoding section 51 performs entropydecoding corresponding to the entropy coding performed in the entropycoding section 17 in the image coding device 10. The entropy decodingsection 51 outputs quantized data DS51 and transform information DS52(corresponding to DS40) obtained by performing the entropy decoding tothe processing selecting switch 52. The entropy decoding section 51 alsooutputs prediction mode information DS53 (corresponding to DS20)obtained by performing the entropy decoding to the predicting section60.

The processing selecting switch 52 performs switching on the basis ofthe transform information DS52 supplied from the entropy decodingsection 51 to output the quantized data DS51 to the first dequantizingsection 53 or the second dequantizing section 54.

The first dequantizing section 53 is configured in a similar manner tothe first dequantizing section 19 in the image coding device 10. Thefirst dequantizing section 53 dequantizes the quantized data DS51supplied via the processing selecting switch 52. In addition, the firstdequantizing section 53 dequantizes the quantized data in each transformblock based on transform block setting information DS76 supplied fromthe transform block setting section 72. The first dequantizing section53 outputs transform coefficients DS54 obtained by performing thedequantization to the first inverse transform section 55.

The first inverse transform section 55 is configured in a similar mannerto the first inverse transform section 21 in the image coding device 10.The first inverse transform section 55 applies an inverse DCT in thehorizontal and vertical directions to the transform coefficients DS54supplied from the first dequantizing section 53 in each transform blockbased on the transform block setting information DS76 supplied from thetransform block setting section 72. The first inverse transform section55 outputs a prediction error signal DS56 obtained by performing theinverse DCT to the arithmetic section 57.

The second dequantizing section 54 is configured in a similar manner tothe second dequantizing section 20 in the image coding device 10. Thesecond dequantizing section 54 dequantizes the quantized data DS51supplied via the processing selecting switch 52 in an oblique directionpattern dequantizing section corresponding to a prediction directionindicated by the transform information DS52. In addition, the seconddequantizing section 54 dequantizes the quantized data in each transformblock based on the transform block setting information DS76 suppliedfrom the transform block setting section 72. The second dequantizingsection 54 outputs transform coefficients DS55 obtained by performingthe dequantization to the second inverse transform section 56.

The second inverse transform section 56 is configured in a similarmanner to the second inverse transform section 22 in the image codingdevice 10. The second inverse transform section 56 performs an inverseDCT of the transform coefficients DS55 supplied from the seconddequantizing section 54 in an oblique direction pattern inverse DCTsection corresponding to the prediction direction indicated by thetransform information DS52. In addition, the second inverse transformsection 56 performs an inverse DCT in each transform block based on thetransform block setting information DS76 supplied from the transformblock setting section 72. The second inverse transform section 56outputs a prediction error signal DS57 obtained by performing theinverse DCT to the arithmetic section 57.

The arithmetic section 57 generates an image signal DS58 by adding apredicted image signal DS61 generated in the predicting section 60 tothe prediction error signal DS56 supplied from the first inversetransform section 55 or the prediction error signal DS57 supplied fromthe second inverse transform section 56. The arithmetic section 57stores the generated image signal DS58 in the reference memory 58.

The image signal DS58 stored in the reference memory 58 is supplied tothe predicting section 60 and the reference image edge detecting section71. In addition, the reference image signal stored in the referencememory 58 is sequentially output as an output image signal DS59 from theimage decoding device 50.

The predicting section 60 performs prediction in a prediction modeindicated by the prediction mode information DS53 using the referenceimage signal DS60 read from the reference memory 58, generates apredicted image signal DS61, and outputs the predicted image signal DS61to the arithmetic section 57.

The reference image edge detecting section 71 is configured in a similarmanner to the reference image edge detecting section 31 in the imagecoding device 10. The reference image edge detecting section 71 detectsan edge using the image signal of the decoded adjacent block stored inthe reference memory 58, and outputs an index DS75 indicating theposition of the edge and the intensity of the edge to the transformblock setting section 72.

The transform block setting section 72 is configured in a similar mannerto the transform block setting section 32 in the image coding device 10.The transform block setting section 72 estimates the continuity of theedge in a block as a decoding object on the basis of the index DS75supplied from the reference image edge detecting section 71 and theprediction mode information DS53 supplied from the entropy decodingsection 51. The transform block setting section 72 sets transform blocksin performing an inverse orthogonal transform and dequantization from aresult of the estimation, and generates the transform block settinginformation DS76 indicating the transform blocks. The transform blocksetting section 72 outputs the generated transform block settinginformation DS76 to the first dequantizing section 53, the seconddequantizing section 54, the first inverse transform section 55, and thesecond inverse transform section 56.

The decoding control section 80 issues control instructions in theprocess of decoding the coded data and the like.

[1-4. Operation of Image Decoding Device]

FIG. 20 is a flowchart of operation of the image decoding device 50 inthe first embodiment. In step ST51, the image decoding device 50 obtainscoded data. The image decoding device 50 obtains the coded data DSC, andstarts decoding in each macroblock or each macroblock pair. The imagedecoding device 50 then proceeds to step ST52.

In step ST52, the image decoding device 50 performs entropy decoding.The image decoding device 50 decodes the variable-length code of eachsyntax of the coded data DSC, and reproduces quantized data DS51,transform information DS52, and prediction mode information DS53. Theimage decoding device 50 then proceeds to step ST53.

In step ST53, the image decoding device 50 performs syntax analysis. Theimage decoding device 50 analyzes the syntaxes from the data obtained byperforming the decoding. The image decoding device 50 then proceeds tostep ST54.

In step ST54, the image decoding device 50 performs initializationrelating to sub-blocks. The image decoding device 50 initializes asub-block index sub_blk to “sub_blk=0” and sets a maximum sub-blocknumber MAX_SUB_BLK at the same time. The image decoding device 50 thenproceeds to step ST55.

In step ST55, the image decoding device 50 determines whether thesub-block index sub_blk is smaller than the maximum sub-block numberMAX_SUB_BLK. When the sub-block index sub_blk is smaller than themaximum sub-block number MAX_SUB_BLK, there is a sub-block yet to bedecoded among sub-blocks within a macroblock, and therefore the imagedecoding device 50 proceeds to step ST56. When the sub-block indexsub_blk is not smaller than the maximum sub-block number MAX_SUB_BLK,there is no sub-block yet to be decoded among the sub-blocks within themacroblock, and therefore the image decoding device 50 proceeds to stepST68.

In step ST56, the image decoding device 50 loads an index mode_idx andtransform information trans_idx. The image decoding device 50 extractsthe index mode_idx and the transform information trans_idx from thecoded data. The image decoding device 50 then proceeds to step ST57.

In step ST57, the image decoding device 50 generates a predicted image.The image decoding device 50 generates a predicted image signal DS61 inthe prediction mode indicated by the index mode_idx using the imagesignal of a reference image, that is, the stored image signal of adecoded adjacent block. The image decoding device 50 then proceeds tostep ST58.

In step ST58, the image decoding device 50 detects an edge. The imagedecoding device 50 detects an edge using the stored image signal of thedecoded adjacent block, and generates an index DS75 indicating theposition of the edge and the intensity of the edge. The image decodingdevice 50 then proceeds to step ST59.

In step ST59, the image decoding device 50 sets transform blocks. Theimage decoding device 50 estimates the continuity of the edge within thesub-block as a decoding object on the basis of the index DS75 indicatingthe position of the edge and the intensity of the edge and theprediction direction of the index mode_idx. Further, the image decodingdevice 50 divides the sub-block on the basis of a result of theestimation of the edge continuity, and sets transform blocks. The imagedecoding device 50 then proceeds to step ST60.

In step ST60, the image decoding device 50 determines whether theprediction mode index mode_idx is smaller than a minimum mode numbermode_direction of an oblique direction prediction mode as a minimumvalue of the mode numbers of the oblique direction prediction modes, orwhether the transform information is “trans_idx=1.” The image decodingdevice 50 proceeds to step ST61 in at least one of the cases where theprediction mode index mode_idx is smaller than the minimum mode numbermode_direction and where the transform information is “trans_idx=1.”Otherwise, the image decoding device 50 proceeds to step ST63.

In step ST61, the image decoding device 50 performs horizontal andvertical dequantization. The image decoding device 50 then proceeds tostep ST62. In step ST62, the image decoding device 50 performs ahorizontal and vertical inverse DCT. The image decoding device 50 thenproceeds to step ST65. The image decoding device 50 for example changesthe processing selecting switch 52 to the side of the first dequantizingsection 53, and performs the dequantization and the inverse DCT usingthe first dequantizing section 53 and the first inverse transformsection 55.

In step ST63, the image decoding device 50 performs oblique directionpattern dequantization. The image decoding device 50 then proceeds tostep ST64. In step ST64, the image decoding device 50 performs anoblique direction pattern inverse DCT. The image decoding device 50 thenproceeds to step ST65. The image decoding device 50 for example changesthe processing selecting switch 52 to the side of the seconddequantizing section 54. The image decoding device 50 changes patternselecting switches in the second dequantizing section 54 and the secondinverse transform section 56 according to the prediction mode indexmode_idx, and performs the dequantization and the inverse DCT using anoblique direction pattern dequantizing section and an oblique directionpattern inverse DCT section corresponding to the prediction direction.

In step ST65, the image decoding device 50 synthesizes a predictionerror and a predicted image. The image decoding device 50 generates animage signal DS58 by adding the predicted image signal DS61 to theprediction error signal DS56 or the prediction error signal DS57. Theimage decoding device 50 then proceeds to step ST66.

In step ST66, the image decoding device 50 stores the generated imagesignal DS58 in the reference memory. The image decoding device 50 storesthe generated image signal DS58 in the reference memory 58. The imagedecoding device 50 then proceeds to step ST67.

In step ST67, the image decoding device 50 adds “1” to the sub-blockindex sub_blk to set a new index sub_blk. The image decoding device 50then returns to step ST55.

In addition, when the image decoding device 50 determines in step ST55that the sub-block index sub_blk is not smaller than the maximumsub-block number MAX_SUB_BLK, and therefore proceeds to step ST68, theimage decoding device 50 outputs a decoded image. The image decodingdevice 50 outputs the image signal stored in the reference memory 58after completion of the decoding of the sub-blocks as the image signalof the decoded image.

Thus, according to the first embodiment, in the image coding device 10,the reference image edge detecting section 31 performs edge detectionusing the image signal of a reference image for a coding object block.In addition, on the basis of a result of the edge detection, thetransform block setting section 32 divides the coding object block suchthat boundaries between blocks after the division do not include anedge, and sets transform blocks. Further, a coding processing sectionformed by the first transform section 13, the second transform section14, the first quantizing section 15, the second quantizing section 16,and the like performs processing including an orthogonal transform ineach transform block, and generates coded data. In addition, in theimage decoding device 50, the reference image edge detecting section 71performs edge detection using the image signal of a reference image fora decoding object block. In addition, on the basis of a result of theedge detection, the transform block setting section 72 divides thedecoding object block such that boundaries between blocks after thedivision do not include an edge, and sets transform blocks equal tothose of the image coding device 10. Further, a decoding processingsection formed by the first dequantizing section 53, the seconddequantizing section 54, the first inverse transform section 55, thesecond inverse transform section 56, and the like performs processingincluding an inverse orthogonal transform in each transform block, andgenerates an image signal. Further, the transform blocks are set inconsideration of edge continuity on the basis of a mode of intra-frameprediction. It is therefore possible to prevent a continuous edge fromstraddling a plurality of transform blocks and thus improve subjectiveimage quality. In addition, transform blocks not including an edge areincreased, so that energy concentration can be improved. Further, thetransform blocks are set to be the same sizes by the same operationperformed in the image coding device and the image decoding device.Thus, even when information on the transform blocks is not included inthe coded data DSC, the coded data DSC can be decoded, so that codingefficiency can be improved.

2. Second Embodiment

Description will next be made of a second embodiment. The foregoingfirst embodiment improves coding efficiency by not including transformblock setting information indicating transform blocks changingadaptively in the coded data. However, the image decoding device needsto be provided with a transform block setting section in order toperform a decoding process correctly even when the transform blocksetting information is not included in the coded data. Accordingly, inthe second embodiment, description will be made of a case in which theconfiguration of the image decoding device is simplified by tolerating aslight decrease in coding efficiency.

[2-1. Configuration of Image Coding Device]

FIG. 21 shows a configuration of an image coding device 10 a accordingto the second embodiment. Incidentally, in FIG. 21, constituent elementscorresponding to those of the image coding device 10 according to thefirst embodiment are identified by the same reference numerals.

The image coding device 10 a includes an arithmetic section 11, aprocessing selecting switch 12, a first transform section 13, a secondtransform section 14, a first quantizing section 15, a second quantizingsection 16, and an entropy coding section 17 a. The image coding device10 a also includes a processing selecting switch 18, a firstdequantizing section 19, a second dequantizing section 20, a firstinverse transform section 21, a second inverse transform section 22, anarithmetic section 23, a reference memory 24, and a predicting section25. The image coding device 10 a further includes a reference image edgedetecting section 31, a transform block setting section 32 a, and acoding control section 40.

The arithmetic section 11 calculates the prediction error of a predictedimage with respect to an input image by subtracting a predicted imagesignal DS18 generated in the predicting section 25 to be described laterfrom an input image signal DS1. The arithmetic section 11 outputs aprediction error signal DS2 indicating the prediction error to theprocessing selecting switch 12.

The processing selecting switch 12 performs switching on the basis oftransform information DS40 supplied from the coding control section 40to output the prediction error signal DS2 to the first quantizingsection 15 or the second quantizing section 16.

The first transform section 13 performs a horizontal and vertical DCT ofthe prediction error signal DS2 supplied from the processing selectingswitch 12. In addition, the first transform section 13 performs ahorizontal and vertical DCT in each transform block based on transformblock setting information DS32 supplied from the transform block settingsection 32 a, and outputs resulting transform coefficients DS3 to thefirst quantizing section 15.

The first quantizing section 15 quantizes the transform coefficients DS3output from the first transform section 13, and outputs quantized dataDS5 to the entropy coding section 17 a and the processing selectingswitch 18. In addition, the first quantizing section 15 quantizes thetransform coefficients DS3 in each transform block on the basis of thetransform block setting information DS32 supplied from the transformblock setting section 32 a.

The second transform section 14 applies a DCT in a prediction directionbased on prediction mode information DS20 from the predicting section 25to the prediction error signal DS2 supplied from the processingselecting switch 12. In addition, the second transform section 14performs a DCT in the prediction direction in each transform block basedon the transform block setting information DS32 supplied from thetransform block setting section 32 a, and outputs resulting transformcoefficients DS4 to the second quantizing section 16.

The second quantizing section 16 quantizes the transform coefficientsDS4 supplied from the second transform section 14 in the predictiondirection on the basis of the prediction mode information DS20 from thepredicting section 25, and outputs quantized data DS6 to the entropycoding section 17 a and the processing selecting switch 18. In addition,the second quantizing section 16 quantizes the transform coefficientsDS4 in each transform block on the basis of the transform block settinginformation DS32 supplied from the transform block setting section 32 a.

The entropy coding section 17 a entropy-codes the quantized data DS5supplied from the first quantizing section 15 or the quantized data DS6supplied from the second quantizing section 16. In addition, the entropycoding section 17 a entropy-codes the prediction mode information DS20generated in the predicting section 25, the transform information DS40generated in the coding control section 40, and the transform blocksetting information DS32 indicating transform blocks set in thetransform block setting section 32 a. The entropy coding section 17 aoutputs coded data DSC obtained by performing the entropy coding.

The processing selecting switch 18 selects an inverse transform methodon the basis of the transform information DS40 supplied from the codingcontrol section 40, and outputs the quantized data DS5 from the firstquantizing section 15 to the first dequantizing section 19 and outputsthe quantized data DS6 from the second quantizing section 16 to thesecond dequantizing section 20.

The first dequantizing section 19 dequantizes the quantized data DS5supplied via the processing selecting switch 18. In addition, the firstdequantizing section 19 dequantizes the quantized data in each transformblock corresponding to that of the first quantizing section 15 on thebasis of the transform block setting information DS32 supplied from thetransform block setting section 32 a. The first dequantizing section 19outputs transform coefficients DS11 obtained by performing thedequantization to the first inverse transform section 21.

The first inverse transform section 21 subjects the transformcoefficients DS11 supplied from the first dequantizing section 19 to aninverse DCT in the horizontal and vertical directions which inverse DCTcorresponds to the DCT in the horizontal and vertical directions in thefirst transform section 13. In addition, the first inverse transformsection 21 performs an inverse DCT in the horizontal and verticaldirections in each transform block based on the transform block settinginformation DS32 supplied from the transform block setting section 32 a.The first inverse transform section 21 outputs a prediction error signalDS13 obtained by performing the inverse DCT to the arithmetic section23.

The second dequantizing section 20 dequantizes the quantized data DS6supplied via the processing selecting switch 18. The second dequantizingsection 20 performs dequantization in the prediction directioncorresponding to that of the second quantizing section 16 on the basisof the prediction mode information DS20 from the predicting section 25.In addition, the second dequantizing section 20 dequantizes thequantized data in each transform block corresponding to that of thesecond quantizing section 16 on the basis of the transform block settinginformation DS32 supplied from the transform block setting section 32 a.The second dequantizing section 20 outputs transform coefficients DS12obtained by performing the dequantization to the second inversetransform section 22.

The second inverse transform section 22 performs an inverse DCT of thetransform coefficients DS12. The second inverse transform section 22performs an inverse DCT in the prediction direction corresponding tothat of the second transform section 14 on the basis of the predictionmode information DS20 from the predicting section 25. In addition, thesecond inverse transform section 22 performs an inverse DCT of thetransform coefficients in each transform block corresponding to that ofthe second transform section 14 on the basis of the transform blocksetting information DS32 supplied from the transform block settingsection 32 a. The second inverse transform section 22 outputs theprediction error signal DS14 obtained by perform the inverse DCT to thearithmetic section 23.

The arithmetic section 23 generates a reference image signal DS15 byadding the predicted image signal DS18 generated in the predictingsection 25 to the prediction error signal DS13 supplied from the firstinverse transform section 21 or the prediction error signal DS14supplied from the second inverse transform section 22. The arithmeticsection 23 stores the generated reference image signal DS15 in thereference memory 24.

The reference image signal DS15 stored in the reference memory 24 issupplied to the predicting section 25 and the reference image edgedetecting section 31.

The predicting section 25 performs intra-frame prediction in eachprediction mode using the reference image signal DS15. In addition, thepredicting section 25 determines a prediction mode that maximizes codingefficiency, and generates prediction mode information DS20 indicatingthe prediction mode that maximizes coding efficiency. The predictingsection 25 outputs the generated prediction mode information DS20 to thesecond transform section 14, the second quantizing section 16, theentropy coding section 17 a, the second dequantizing section 20, thesecond inverse transform section 22, and the transform block settingsection 32 a. Further, the predicting section 25 generates the predictedimage signal DS18 in the prediction mode that maximizes codingefficiency, and outputs the predicted image signal DS18 to thearithmetic sections 11 and 23.

The reference image edge detecting section 31 detects an edge using animage signal of a coded adjacent bock stored in the reference memory 24,and outputs an index DS31 indicating the position of the edge and theintensity of the edge to the transform block setting section 32 a.

The transform block setting section 32 a estimates the continuity of theedge within a sub-block as a coding object on the basis of the indexDS31 supplied from the reference image edge detecting section 31 and theprediction mode information DS20 supplied from the predicting section25. The transform block setting section 32 a sets transform blocks in anorthogonal transform and quantization from a result of the estimation,and generates transform block setting information DS32 indicating theset transform blocks. The transform block setting section 32 a outputsthe generated transform block setting information DS32 to the firsttransform section 13, the second transform section 14, the firstquantizing section 15, the second quantizing section 16, the firstdequantizing section 19, the second dequantizing section 20, the firstinverse transform section 21, and the second inverse transform section22. In addition, the transform block setting section 32 a outputs thegenerated transform block setting information DS32 to the entropy codingsection 17 a.

The coding control section 40 generates the transform information DS40,and outputs the transform information DS40 to the processing selectingswitch 12, the entropy coding section 17 a, and the processing selectingswitch 18.

[2-2. Operation of Image Coding Device]

The image coding device 10 a according to the second embodiment performsthe process of the flowchart shown in FIGS. 13 and 14 to generate andoutput coded data. In addition, the image coding device 10 a includesthe transform block setting information DS32 for an orthogonal transformin the optimum mode in the data on the optimum mode in the process ofloading the data on the optimum mode in step ST22 in FIG. 14. Forexample, the image coding device 10 a entropy-codes the transform blocksetting information DS32, and thereafter includes the transform blocksetting information DS32 as a header in the coded data DSC.

[2-3. Configuration of Image Decoding Device]

Description will next be made of an image decoding device that decodesthe coded data generated by the image coding device 10 a.

FIG. 22 shows a configuration of an image decoding device 50 a in thesecond embodiment. Incidentally, in FIG. 22, constituent elementscorresponding to those of the image decoding device 50 according to thefirst embodiment are identified by the same reference numerals.

The image decoding device 50 a includes an entropy decoding section 51a, a processing selecting switch 52, a first dequantizing section 53, asecond dequantizing section 54, a first inverse transform section 55, asecond inverse transform section 56, an arithmetic section 57, areference memory 58, and a predicting section 60. The image decodingdevice 50 a also includes a decoding control section 80.

The entropy decoding section 51 a entropy-decodes the coded data DSCreceived as input. The entropy decoding section 51 a performs entropydecoding corresponding to the entropy coding performed in the entropycoding section 17 a in the image coding device 10 a. The entropydecoding section 51 a outputs quantized data DS51 and transforminformation DS52 (corresponding to DS40) obtained by performing theentropy decoding to the processing selecting switch 52. The entropydecoding section 51 a also outputs prediction mode information DS53(corresponding to DS20) obtained by performing the entropy decoding tothe second dequantizing section 54, the second inverse transform section56, and the predicting section 60. Further, the entropy decoding section51 a outputs transform block setting information DS76 (corresponding toDS32) obtained by performing the entropy decoding to the firstdequantizing section 53, the second dequantizing section 54, the firstinverse transform section 55, and the second inverse transform section56.

The processing selecting switch 52 performs switching on the basis ofthe transform information DS52 supplied from the entropy decodingsection 51 a to output the quantized data DS51 to the first dequantizingsection 53 or the second dequantizing section 54.

The first dequantizing section 53 is configured in a similar manner tothe first dequantizing section 19 in the image coding device 10 a. Thefirst dequantizing section 53 dequantizes the quantized data DS51supplied via the processing selecting switch 52. In addition, the firstdequantizing section 53 dequantizes the quantized data in each blockbased on transform block setting information DS76 supplied from theentropy decoding section 51 a. The first dequantizing section 53 outputstransform coefficients DS54 obtained by performing the dequantization tothe first inverse transform section 55.

The first inverse transform section 55 is configured in a similar mannerto the first inverse transform section 21 in the image coding device 10a. The first inverse transform section 55 applies an inverse DCT in thehorizontal and vertical directions to the transform coefficients DS54supplied from the first dequantizing section 53 in each block based onthe transform block setting information DS76 supplied from the entropydecoding section 51 a. The first inverse transform section 55 outputs aprediction error signal DS56 obtained by performing the inverse DCT tothe arithmetic section 57.

The second dequantizing section 54 is configured in a similar manner tothe second dequantizing section 20 in the image coding device 10 a. Thesecond dequantizing section 54 dequantizes the quantized data DS51supplied via the processing selecting switch 52. In addition, the seconddequantizing section 54 dequantizes the quantized data in each blockbased on the transform block setting information DS76 supplied from theentropy decoding section 51 a. The second dequantizing section 54outputs transform coefficients DS55 obtained by performing thedequantization to the second inverse transform section 56.

The second inverse transform section 56 is configured in a similarmanner to the second inverse transform section 22 in the image codingdevice 10 a. The second inverse transform section 56 applies an inverseDCT according to a prediction direction to the transform coefficientsDS55 supplied from the second dequantizing section 54 in each blockbased on the transform block setting information DS76 supplied from theentropy decoding section 51 a. The second inverse transform section 56outputs a prediction error signal DS57 obtained by performing theinverse DCT to the arithmetic section 57.

The arithmetic section 57 generates an image signal DS58 by adding apredicted image signal DS61 generated in the predicting section 60 tothe prediction error signal DS56 supplied from the first inversetransform section 55 or the prediction error signal DS57 supplied fromthe second inverse transform section 56. The arithmetic section 57stores the generated image signal DS58 in the reference memory 58.

The image signal DS58 stored in the reference memory 58 is supplied tothe predicting section 60 as a reference image signal DS60. In addition,the reference image signal stored in the reference memory 58 is outputas an output image signal DS59 from the image decoding device 50 a.

The predicting section 60 performs prediction in a prediction modeindicated by the prediction mode information DS53 using the referenceimage signal DS60 read from the reference memory 58, generates apredicted image signal DS61, and outputs the predicted image signal DS61to the arithmetic section 57.

The decoding control section 80 issues control instructions in theprocess of decoding the coded data and the like.

[2-4. Operation of Image Decoding Device]

FIG. 23 is a flowchart of operation of the image decoding device 50 a inthe second embodiment. In step ST51, the image decoding device 50 aobtains coded data. The image decoding device 50 a obtains the codeddata DSC, and starts decoding in each macroblock or each macroblockpair. The image decoding device 50 a then proceeds to step ST52 a.

In step ST52 a, the image decoding device 50 a performs entropydecoding. The image decoding device 50 a decodes the variable-lengthcode of each syntax of the coded data DSC, and reproduces quantized dataDS51, transform information DS52, prediction mode information DS53, andtransform block setting information DS76. The image decoding device 50 athen proceeds to step ST53.

In step ST53, the image decoding device 50 a performs syntax analysis.The image decoding device 50 a analyzes the syntaxes from the dataobtained by performing the decoding. The image decoding device 50 a thenproceeds to step ST54.

In step ST54, the image decoding device 50 a performs initializationrelating to sub-blocks. The image decoding device 50 a initializes asub-block index sub_blk to “sub_blk=0” and sets a maximum sub-blocknumber MAX_SUB_BLK at the same time. The image decoding device 50 a thenproceeds to step ST55.

In step ST55, the image decoding device 50 a determines whether thesub-block index sub_blk is smaller than the maximum sub-block numberMAX_SUB_BLK. When the sub-block index sub_blk is smaller than themaximum sub-block number MAX_SUB_BLK, there is a sub-block yet to bedecoded among sub-blocks within a macroblock, and therefore the imagedecoding device 50 a proceeds to step ST56 a. When the sub-block indexsub_blk is not smaller than the maximum sub-block number MAX_SUB_BLK,there is no sub-block yet to be decoded among the sub-blocks within themacroblock, and therefore the image decoding device 50 a proceeds tostep ST68.

In step ST56 a, the image decoding device 50 a loads an index mode_idx,transform information trans_idx, and transform block setting informationDS76. The image decoding device 50 a extracts the index mode_idx, thetransform information trans_idx, and the transform block settinginformation DS76 from the coded data. The image decoding device 50 athen proceeds to step ST57.

In step ST57, the image decoding device 50 a generates a predictedimage. The image decoding device 50 a generates a predicted image signalDS61 in the prediction mode indicated by the index mode_idx using theimage signal of a reference image, that is, the stored image signal of adecoded adjacent block. The image decoding device 50 a then proceeds tostep ST60.

In step ST60, the image decoding device 50 a determines whether theprediction mode index mode_idx is smaller than a minimum mode numbermode_direction of an oblique direction prediction mode as a minimumvalue of the mode numbers of the oblique direction prediction modes, orwhether the transform information is “trans_idx=1.” The image decodingdevice 50 a proceeds to step ST61 in at least one of the cases where theprediction mode index mode_idx is smaller than the minimum mode numbermode_direction and where the transform information is “trans_idx=1.”Otherwise, the image decoding device 50 a proceeds to step ST63.

In step ST61, the image decoding device 50 a performs horizontal andvertical dequantization. The image decoding device 50 a then proceeds tostep ST62. In step ST62, the image decoding device 50 a performs ahorizontal and vertical inverse DCT. The image decoding device 50 a thenproceeds to step ST65. The image decoding device 50 a for examplechanges the processing selecting switch 52 to the side of the firstdequantizing section 53, and performs the dequantization and the inverseDCT using the first dequantizing section 53 and the first inversetransform section 55.

In step ST63, the image decoding device 50 a performs oblique directionpattern dequantization. The image decoding device 50 a then proceeds tostep ST64. In step ST64, the image decoding device 50 a performs anoblique direction pattern inverse DCT. The image decoding device 50 athen proceeds to step ST65. The image decoding device 50 a for examplechanges the processing selecting switch 52 to the side of the seconddequantizing section 54. The image decoding device 50 a changes patternselecting switches in the second dequantizing section 54 and the secondinverse transform section 56 according to the prediction mode indexmode_idx, and performs the dequantization and the inverse DCT using anoblique direction pattern dequantizing section and an oblique directionpattern inverse DCT section corresponding to the prediction direction.

In step ST65, the image decoding device 50 a synthesizes a predictionerror and a predicted image. The image decoding device 50 a generates animage signal DS58 by adding the predicted image signal DS61 to theprediction error signal DS56 or the prediction error signal DS57. Theimage decoding device 50 a then proceeds to step ST66.

In step ST66, the image decoding device 50 a stores the generated imagesignal DS58 in the reference memory. The image decoding device 50 astores the generated image signal DS58 in the reference memory 58. Theimage decoding device 50 a then proceeds to step ST67.

In step ST67, the image decoding device 50 a adds “1” to the sub-blockindex sub_blk to set a new index sub_blk. The image decoding device 50 athen returns to step ST55.

In addition, when the image decoding device 50 a determines in step ST55that the sub-block index sub_blk is not smaller than the maximumsub-block number MAX_SUB_BLK, and therefore proceeds to step ST68, theimage decoding device 50 a outputs a decoded image. The image decodingdevice 50 a outputs the image signal stored in the reference memory 58after completion of the decoding of the sub-blocks as the image signalof the decoded image.

Thus, according to the second embodiment, as in the first embodiment,transform blocks are set in consideration of edge continuity on thebasis of a mode of intra-frame prediction. It is therefore possible toprevent a continuous edge from straddling a plurality of transformblocks and thus improve subjective image quality. In addition, transformblocks not including an edge are increased, so that energy concentrationcan be improved. Further, because the transform block settinginformation on the transform blocks is output in a state of beingmultiplexed in the coded data, the image decoding device 50 a does notneed to have a reference image edge detecting section for performingedge detection or a transform block setting section for setting thetransform blocks on the basis of a result of the edge detection. Theconfiguration of the image decoding device can be simplified as comparedwith the first embodiment.

3. Third Embodiment

The foregoing first and second embodiments set transform blocks on thebasis of a result of estimation of the continuity of an edge within asub-block as a coding object from the edge in an adjacent block. Thatis, transform blocks are set using an image in a spatial direction as areference image. However, the reference image is not limited to thespatial direction, but an image in a temporal direction can also be usedas a reference image. Description will next be made of a case of usingan image in the temporal direction as a reference image as a thirdembodiment.

[3-1. Configuration of Image Coding Device]

FIG. 24 shows a configuration of an image coding device in the thirdembodiment. Incidentally, in FIG. 24, constituent elements correspondingto those of the image coding device 10 according to the first embodimentare identified by the same reference numerals.

An image coding device 10 b includes an arithmetic section 11, aprocessing selecting switch 12, a first transform section 13, a secondtransform section 14, a first quantizing section 15, a second quantizingsection 16, and an entropy coding section 17 b. The image coding device10 b also includes a processing selecting switch 18, a firstdequantizing section 19, a second dequantizing section 20, a firstinverse transform section 21, a second inverse transform section 22, anarithmetic section 23, a reference memory 24, a predicting section 25, areference image edge detecting section 31, and a transform block settingsection 32. The image coding device 10 b further includes an imagesignal selecting switch 30, a motion estimating section 35 havingfunctions equivalent to those of MPEG and the H.264/AVC system, a motioncompensating section 36, a prediction selecting switch 37, and a codingcontrol section 40.

The arithmetic section 11 calculates the prediction error of a predictedimage with respect to an input image by subtracting a predicted imagesignal DS37 as an output of the prediction selecting switch 37 to bedescribed later from an input image signal DS1. The arithmetic section11 outputs a prediction error signal DS2 indicating the prediction errorto the processing selecting switch 12.

The processing selecting switch 12 performs switching on the basis oftransform information DS40 supplied from the coding control section 40to output the prediction error signal DS2 to the first quantizingsection 15 or the second quantizing section 16.

The first transform section 13 performs a horizontal and vertical DCT ofthe prediction error signal DS2 supplied from the processing selectingswitch 12. In addition, the first transform section 13 performs ahorizontal and vertical DCT in each transform block based on transformblock setting information DS32 supplied from the transform block settingsection 32, and outputs resulting transform coefficients DS3 to thefirst quantizing section 15.

The first quantizing section 15 quantizes the transform coefficients DS3output from the first transform section 13, and outputs quantized dataDS5 to the entropy coding section 17 b and the processing selectingswitch 18. In addition, the first quantizing section 15 quantizes thetransform coefficients DS3 in each transform block on the basis of thetransform block setting information DS32 supplied from the transformblock setting section 32.

The second transform section 14 applies a DCT in a prediction directionbased on prediction mode information DS20 from the predicting section 25to the prediction error signal DS2 supplied from the processingselecting switch 12. In addition, the second transform section 14performs a DCT in the prediction direction in each transform block basedon the transform block setting information DS32 supplied from thetransform block setting section 32, and outputs resulting transformcoefficients DS4 to the second quantizing section 16.

The second quantizing section 16 quantizes the transform coefficientsDS4 supplied from the second transform section 14 in the predictiondirection on the basis of the prediction mode information DS20 from thepredicting section 25, and outputs quantized data DS6 to the entropycoding section 17 b and the processing selecting switch 18. In addition,the second quantizing section 16 quantizes the transform coefficientsDS4 in each transform block on the basis of the transform block settinginformation DS32 supplied from the transform block setting section 32.

The entropy coding section 17 b entropy-codes the quantized data DS5supplied from the first quantizing section 15 or the quantized data DS6supplied from the second quantizing section 16. In addition, the entropycoding section 17 b entropy-codes the prediction mode information DS20generated in the predicting section 25, the transform information DS40generated in the coding control section 40, and motion vectorinformation DS35 detected in the motion estimating section 35. Theentropy coding section 17 b outputs coded data DSC obtained byperforming the entropy coding.

The processing selecting switch 18 selects an inverse transform methodon the basis of the transform information DS40 supplied from the codingcontrol section 40, and outputs the quantized data DS5 from the firstquantizing section 15 to the first dequantizing section 19 and outputsthe quantized data DS6 from the second quantizing section 16 to thesecond dequantizing section 20.

The first dequantizing section 19 dequantizes the quantized data DS5supplied via the processing selecting switch 18. In addition, the firstdequantizing section 19 dequantizes the quantized data in each transformblock corresponding to that of the first quantizing section 15 on thebasis of the transform block setting information DS32 supplied from thetransform block setting section 32. The first dequantizing section 19outputs transform coefficients DS11 obtained by performing thedequantization to the first inverse transform section 21.

The first inverse transform section 21 subjects the transformcoefficients DS11 supplied from the first dequantizing section 19 to aninverse DCT in the horizontal and vertical directions which inverse DCTcorresponds to the DCT in the horizontal and vertical directions in thefirst transform section 13. In addition, the first inverse transformsection 21 performs an inverse DCT in the horizontal and verticaldirections in each transform block based on the transform block settinginformation DS32 supplied from the transform block setting section 32.The first inverse transform section 21 outputs a prediction error signalDS13 obtained by performing the inverse DCT to the arithmetic section23.

The second dequantizing section 20 dequantizes the quantized data DS6supplied via the processing selecting switch 18. The second dequantizingsection 20 performs dequantization in the prediction directioncorresponding to that of the second quantizing section 16 on the basisof the prediction mode information DS20 from the predicting section 25.In addition, the second dequantizing section 20 dequantizes thequantized data in each transform block corresponding to that of thesecond quantizing section 16 on the basis of the transform block settinginformation DS32 supplied from the transform block setting section 32.The second dequantizing section 20 outputs transform coefficients DS12obtained by performing the dequantization to the second inversetransform section 22.

The second inverse transform section 22 performs an inverse DCT of thetransform coefficients DS12. The second inverse transform section 22performs an inverse DCT in the prediction direction corresponding tothat of the second transform section 14 on the basis of the predictionmode information DS20 from the predicting section 25. In addition, thesecond inverse transform section 22 performs an inverse DCT of thetransform coefficients in each transform block corresponding to that ofthe second transform section 14 on the basis of the transform blocksetting information DS32 supplied from the transform block settingsection 32. The second inverse transform section 22 outputs theprediction error signal DS14 obtained by performing the inverse DCT tothe arithmetic section 23.

The arithmetic section 23 generates a reference image signal DS15 byadding the predicted image signal DS37 output from the predictionselecting switch 37 to the prediction error signal DS13 supplied fromthe first inverse transform section 21 or the prediction error signalDS14 supplied from the second inverse transform section 22. Thearithmetic section 23 stores the generated reference image signal DS15in the reference memory 24.

The reference memory 24 stores not only the image signal of blocksadjacent in the spatial direction but also images in the temporaldirection, that is, images of a plurality of frames as reference images.Incidentally, the images of the plurality of frames are stored afterbeing subjected to deblocking filter processing.

The image signal DS16 of the adjacent blocks which image signal is readfrom the reference memory 24 is supplied to the predicting section 25and the image signal selecting switch 30. In addition, the image signalDS17 of the frame images which image signal is read from the referencememory 24 is supplied to the motion estimating section 35 and the motioncompensating section 36.

The predicting section 25 performs intra-frame prediction in eachprediction mode using the image signal DS16. In addition, the predictingsection 25 determines a prediction mode that maximizes codingefficiency, and generates prediction mode information DS20 indicatingthe prediction mode that maximizes coding efficiency. The predictingsection 25 outputs the generated prediction mode information DS20 to thesecond transform section 14, the second quantizing section 16, theentropy coding section 17 b, the second dequantizing section 20, thesecond inverse transform section 22, and the transform block settingsection 32. Further, the predicting section 25 generates a predictedimage signal DS18 in the prediction mode that maximizes codingefficiency, and outputs the predicted image signal DS18 to theprediction selecting switch 37.

The image signal selecting switch 30 selects one of the image signalDS16 supplied from the reference memory 24 and a predicted image signalDS36 supplied from the motion compensating section 36 on the basis ofthe transform information DS40, and outputs the selected image signal tothe reference image edge detecting section 31. For example, in a case ofan intra-frame prediction mode, the image signal selecting switch 30selects the image signal DS16 supplied from the reference memory 24, andoutputs the selected image signal to the reference image edge detectingsection 31. In a case of an inter-frame prediction mode, the imagesignal selecting switch 30 selects the predicted image signal DS36supplied from the motion compensating section 36, and outputs theselected image signal to the reference image edge detecting section 31.

The reference image edge detecting section 31 detects an edge using theimage signal selected by the image signal selecting switch 30, andoutputs an index DS31 indicating the position of the edge and theintensity of the edge to the transform block setting section 32.

The transform block setting section 32 estimates the continuity of theedge within a sub-block as a coding object on the basis of the indexDS31 supplied from the reference image edge detecting section 31 and theprediction mode information DS20 supplied from the predicting section25. The transform block setting section 32 sets transform blocks in anorthogonal transform and quantization from a result of the estimation,and generates transform block setting information DS32 indicating theset transform blocks. The transform block setting section 32 outputs thegenerated transform block setting information DS32 to the firsttransform section 13, the second transform section 14, the firstquantizing section 15, the second quantizing section 16, the firstdequantizing section 19, the second dequantizing section 20, the firstinverse transform section 21, and the second inverse transform section22.

The motion estimating section 35 performs motion estimation in thesub-block and detects a motion vector using the input image signal DS1and the image signal DS17 supplied from the reference memory 24. Themotion estimating section 35 outputs motion vector information DS35indicating the detected motion vector to the motion compensating section36 and the entropy coding section 17 b.

The motion compensating section 36 applies motion compensation based onthe motion vector information DS35 supplied from the motion estimatingsection 35 to a reference image based on the image signal DS17 suppliedfrom the reference memory 24. The motion compensating section 36 outputsa predicted image signal DS36 in an inter-frame prediction mode whichimage signal is generated by the motion compensation to the predictionselecting switch 37 and the image signal selecting switch 30.

The prediction selecting switch 37 selects one of the predicted imagesignal DS18 supplied from the predicting section 25 and the predictedimage signal DS36 supplied from the motion compensating section 36 onthe basis of the transform information DS40, and outputs the selectedimage signal to the arithmetic sections 11 and 23. For example, in acase of an intra-frame prediction mode, the prediction selecting switch37 selects the predicted image signal DS18 supplied from the predictingsection 25, and outputs the selected image signal as predicted imagesignal DS37 to the arithmetic sections 11 and 23. In a case of aninter-frame prediction mode, the prediction selecting switch 37 selectsthe predicted image signal DS36 supplied from the motion compensatingsection 36, and outputs the selected image signal as predicted imagesignal DS37 to the arithmetic sections 11 and 23.

The coding control section 40 generates transform information DS40. Thetransform information DS40 is information for selecting either a processof performing a horizontal and vertical DCT in relation to an orthogonaltransform and horizontal and vertical quantization or a process ofperforming a one-dimensional DCT and quantization along a predictiondirection indicated by the prediction mode information DS20. Inaddition, suppose that the transform information DS40 in the thirdembodiment indicates which of intra-frame prediction and inter-frameprediction is selected. The coding control section 40 outputs thegenerated transform information DS40 to the processing selecting switch12, the entropy coding section 17 b, the processing selecting switch 18,the image signal selecting switch 30 and the prediction selecting switch37.

[3-2. Operation of Image Coding Device]

FIG. 25 and FIG. 26 are flowcharts of operation of the image codingdevice 10 b in the third embodiment. In step ST101, the image codingdevice 10 b obtains an input image. The image coding device 10 b obtainsan input image signal DS1, and starts coding in each macroblock or eachmacroblock pair.

In step ST102, the image coding device 10 b performs initializationrelating to sub-blocks. The image coding device 10 b initializes asub-block index sub_blk to “sub_blk=0” and sets a maximum sub-blocknumber MAX_SUB_BLK at the same time. The image coding device 10 b thenproceeds to step ST103.

In step ST103, the image coding device 10 b determines whether thesub-block index sub_blk is smaller than the maximum sub-block numberMAX_SUB_BLK. When the sub-block index sub_blk is smaller than themaximum sub-block number MAX_SUB_BLK, there is a sub-block yet to becoded among sub-blocks within a macroblock, and therefore the imagecoding device 10 b proceeds to step ST104. When the sub-block indexsub_blk is not smaller than the maximum sub-block number MAX_SUB_BLK,there is no sub-block yet to be coded among the sub-blocks within themacroblock, and therefore the image coding device 10 b proceeds to stepST125.

In step ST104, the image coding device 10 b sets a motion vector searchposition MV_x in an X-direction (for example a horizontal direction) anda motion vector search position MV_y in a Y-direction (for example avertical direction) to search start points START_X and START_Y. Theimage coding device 10 b then proceeds to step ST105.

In step ST105, the image coding device 10 b determines whetherMV_x<END_X and MV_y<END_Y. When the search positions are within searchranges up to search end points END_X and END_Y, the image coding device10 b proceeds to step ST106. When the search positions exceed to thesearch ranges, the image coding device 10 b proceeds to step ST123.

In step ST106, the image coding device 10 b performs initializationrelating to prediction directions. The image coding device 10 binitializes an index mode_idx_d to “mode_idx_d=0,” and sets a maximumselectable direction number MAX_MODE_d. The index mode_idx_d indicates aprediction direction, and corresponds to a prediction direction in anintra-frame prediction mode. The maximum direction number MAX_MODE_dcorresponds to the number of selectable prediction directions, that is,the maximum mode number MAX_MODE of intra-frame prediction modes. Thususing the index mode_idx_d indicating a prediction direction makes itpossible to perform DCTs corresponding to oblique prediction directionsand detect an optimum mode also in inter-frame prediction.

In step ST107, the image coding device 10 b determines whether theprediction direction index mode_idx_d is smaller than the maximumdirection number MAX_MODE_d. When the prediction direction indexmode_idx_d is smaller than the maximum direction number MAX_MODE_d, theimage coding device 10 b determines that not all prediction directionshave been tried. The image coding device 10 b then proceeds to stepST108. When the prediction direction index mode_idx_d is not smallerthan the maximum direction number MAX_MODE_d, the image coding device 10b determines that all the prediction directions have been tried. Theimage coding device 10 b then proceeds to step ST122.

In step ST108, the image coding device 10 b sets transform informationtrans_idx. The image coding device 10 b sets the transform informationtrans_idx according to the value of the prediction direction indexmode_idx_d. For example, when the value of the prediction directionindex mode_idx_d indicates an oblique prediction direction, the imagecoding device 10 b sets the transform information trans_idx to“trans_idx=0.” The image coding device 10 b then proceeds to step ST109.When the value of the prediction direction index mode_idx_d indicates anon-oblique prediction direction, the image coding device 10 b sets thetransform information trans_idx to “trans_idx=1.” The image codingdevice 10 b then proceeds to step ST109.

In step ST109, the image coding device 10 b generates amotion-compensated signal at the search positions MV_x and MV_y from areference frame. The image coding device 10 b then proceeds to stepST110.

In step ST110, the image coding device 10 b detects an edge using thegenerated motion-compensated signal, and generates an index DS31indicating the position of the edge and the intensity of the edge. Theimage coding device 10 b then proceeds to step ST111.

In step ST111, the image coding device 10 b sets transform blocks. Theimage coding device 10 b estimates the continuity of the edge within thesub-block as the coding object on the basis of the index DS31 indicatingthe position of the edge and the intensity of the edge. Further, theimage coding device 10 b sets transform blocks on the basis of a resultof the estimation of the continuity of the edge. The image coding device10 b then proceeds to step ST112.

In step ST112, the image coding device 10 b determines whether theprediction direction index mode_idx_d is smaller than a minimumdirection number mode_direction_d as a minimum index value of theoblique prediction directions, or whether the transform information is“trans_idx=1.” In at least one of the cases where the predictiondirection index mode_idx_d is smaller than the minimum direction numbermode_direction_d and where the transform information is “trans_idx=1,”the image coding device 10 b proceeds to step ST113. Otherwise, theimage coding device 10 b proceeds to step ST115.

In step ST113, the image coding device 10 b performs a horizontal andvertical DCT. The image coding device 10 b then proceeds to step ST114.In step ST114, the image coding device 10 b performs horizontal andvertical quantization. The image coding device 10 b then proceeds tostep ST117. The image coding device 10 b for example changes theprocessing selecting switch 12 to the side of the first transformsection 13, and performs the DCT and the quantization using the firsttransform section 13 and the first quantizing section 15.

In step ST115, the image coding device 10 b performs an obliquedirection pattern DCT. The image coding device 10 b then proceeds tostep ST116. In step ST116, the image coding device 10 b performs obliquedirection pattern quantization. The image coding device 10 b thenproceeds to step ST117. The image coding device 10 b for example changesthe processing selecting switch 12 to the side of the second transformsection 14. In addition, the image coding device 10 b changes a patternselecting switch 140 in the second transform section 14 and a patternselecting switch 160 in the second quantizing section 16 according tothe prediction direction index mode_idx_d. The image coding device 10 bperforms the DCT and the quantization using an oblique direction patternDCT section and an oblique direction pattern quantizing sectioncorresponding to the prediction direction.

In step ST117, the image coding device 10 b performs entropy coding. Theimage coding device 10 b entropy-codes the quantized data DS5 and DS6,the prediction mode information DS20, the motion vector informationDS35, and the transform information DS40 in the entropy coding section17 b. The image coding device 10 b then proceeds to step ST118.

In step ST118, the image coding device 10 b stores coding cost. Thecoding control section 40 in the image coding device 10 b calculates acost value K as described above, and stores the calculated cost value K.The image coding device 10 b then proceeds to step ST119.

In step ST119, the image coding device 10 b determines whether thetransform information trans_idx is “trans_idx=0.” When the transforminformation trans_idx is “trans_idx=0,” the image coding device 10 bproceeds to step ST120. When the transform information trans_idx is not“trans_idx=0,” the image coding device 10 b proceeds to step ST121.

In step ST120, the image coding device 10 b adds “1” to the transforminformation trans_idx to set new transform information trans_idx. Theimage coding device 10 b then returns to step ST112.

In step ST121, the image coding device 10 b adds “1” to the predictiondirection index mode_idx_d to set a new index mode_idx_d. The imagecoding device 10 b then returns to step ST107.

When thereafter determining in step ST107 that the prediction directionindex mode_idx_d is not smaller than the maximum direction numberMAX_MODE_d and thus proceeding to step ST122, the image coding device 10b changes the search position MV_x or the search position MV_y to a newposition. The image coding device 10 b then returns to step ST105.

When the search positions exceed the search ranges in step ST105, theimage coding device 10 b proceeds to step ST123, where the image codingdevice 10 b adds “1” to the sub-block index sub_blk to set a new indexsub_blk. The image coding device 10 b then returns to step ST103.

When determining in step ST103 that the sub-block index sub_blk is notsmaller than the maximum sub-block number MAX_SUB_BLK, the image codingdevice 10 b proceeds to step ST125.

The image coding device 10 b performs not only the inter-frameprediction but also the intra-frame prediction described with referenceto FIGS. 13 and 14 in step ST124. The image coding device 10 b thenproceeds to step ST125.

In step ST125, the image coding device 10 b loads data on an optimummode. The image coding device 10 b compares stored coding costs witheach other, and determines that a mode or a prediction directionproviding highest coding efficiency in the intra-frame prediction andthe inter-frame prediction is an optimum mode. The image coding device10 b loads data on the determined optimum mode. The image coding device10 b then proceeds to step ST126.

In step ST126, the image coding device 10 b multiplexes and sends outcoded data obtained by performing coding in the optimum mode in themacroblock. In addition, the image coding device 10 b entropy-codes theprediction mode information DS20, that is, the index mode_idx of theoptimum mode and the transform information DS40, that is, the transforminformation trans_idx, and includes the entropy-coded information in thecoded data. Further, the image coding device 10 b entropy-codes themotion vector information DS35 in the optimum mode, and includes theentropy-coded motion vector information DS35 in the coded data.

In a case of intra-frame prediction, the reference image edge detectingsection 31 of the image coding device 10 b performs similar operation tothat in the first embodiment. In a case of inter-frame prediction, thereference image edge detecting section 31 performs edge detection usinga motion-compensated signal obtained by performing motion compensationas in MPEG or the H.264/AVC system. FIG. 27A shows edges detected byusing a motion-compensated signal. The image coding device 10 b performsedge detection using for example a Sobel filter as described above, andobtains indexes indicating the positions of the edges and theintensities of the edges.

In a case of intra-frame prediction, the transform block setting section32 performs similar operation to that in the first embodiment. In a caseof inter-frame prediction, the transform block setting section 32 setstransform blocks on the basis of an index indicating the position of anedge and the intensity of the edge in a motion-compensated signal. Forexample, when edges are detected as shown in FIG. 27A, the transformblock setting section 32 estimates that the same edges are present inthe sub-block as the coding object as shown in FIG. 27B, and setstransform blocks as in a case of intra-frame prediction on the basis ofa result of the estimation.

[3-3. Configuration of Image Decoding Device]

Description will next be made of an image decoding device that decodesthe coded data generated by the image coding device 10 b.

FIG. 28 shows a configuration of an image decoding device 50 b in thethird embodiment. Incidentally, in FIG. 28, constituent elementscorresponding to those of the image decoding device 50 according to thefirst embodiment are identified by the same reference numerals.

The image decoding device 50 b includes an entropy decoding section 51b, a processing selecting switch 52, a first dequantizing section 53, asecond dequantizing section 54, a first inverse transform section 55, asecond inverse transform section 56, an arithmetic section 57, areference memory 58, a predicting section 60, a motion compensatingsection 61, and a prediction selecting switch 62. The image decodingdevice 50 b also includes an image signal selecting switch 70, areference image edge detecting section 71, a transform block settingsection 72, and a decoding control section 80.

The entropy decoding section 51 b entropy-decodes the coded data DSCreceived as input. The entropy decoding section 51 b performs entropydecoding corresponding to the entropy coding performed in the entropycoding section 17 b in the image coding device 10 b. The entropydecoding section 51 b outputs quantized data DS51 and transforminformation DS52 (corresponding to DS40) obtained by performing theentropy decoding to the processing selecting switch 52. The entropydecoding section 51 b also outputs prediction mode information DS53(corresponding to DS20) obtained by performing the entropy decoding tothe second quantizing section 54, second inverse transform section 56,the predicting section 60 and the transform block setting section 72.

The processing selecting switch 52 performs switching on the basis ofthe transform information DS52 supplied from the entropy decodingsection 51 b to output the quantized data DS51 to the first dequantizingsection 53 or the second dequantizing section 54.

The first dequantizing section 53 is configured in a similar manner tothe first dequantizing section 19 in the image coding device 10 b. Thefirst dequantizing section 53 dequantizes the quantized data DS51supplied via the processing selecting switch 52. In addition, the firstdequantizing section 53 dequantizes the quantized data in each transformblock based on transform block setting information DS76 supplied fromthe transform block setting section 72. The first dequantizing section53 outputs transform coefficients DS54 obtained by performing thedequantization to the first inverse transform section 55.

The first inverse transform section 55 is configured in a similar mannerto the first inverse transform section 21 in the image coding device 10b. The first inverse transform section 55 applies an inverse DCT in thehorizontal and vertical directions to the transform coefficients DS54supplied from the first dequantizing section 53 in each transform blockbased on the transform block setting information DS76 supplied from thetransform block setting section 72. The first inverse transform section55 outputs a prediction error signal DS56 obtained by performing theinverse DCT to the arithmetic section 57.

The second dequantizing section 54 is configured in a similar manner tothe second dequantizing section 20 in the image coding device 10 b. Thesecond dequantizing section 54 dequantizes the quantized data DS51supplied via the processing selecting switch 52. In addition, the seconddequantizing section 54 dequantizes the quantized data in each transformblock based on the transform block setting information DS76 suppliedfrom the transform block setting section 72. The second dequantizingsection 54 outputs transform coefficients DS55 obtained by performingthe dequantization to the second inverse transform section 56.

The second inverse transform section 56 is configured in a similarmanner to the second inverse transform section 22 in the image codingdevice 10 b. The second inverse transform section 56 applies an inverseDCT according to a prediction direction to the transform coefficientsDS55 supplied from the second dequantizing section 54 in each transformblock based on the transform block setting information DS76 suppliedfrom the transform block setting section 72. The second inversetransform section 56 outputs a prediction error signal DS57 obtained byperforming the inverse DCT to the arithmetic section 57.

The arithmetic section 57 generates an image signal DS58 by adding apredicted image signal DS73 supplied from the prediction selectingswitch 62 to the prediction error signal DS56 supplied from the firstinverse transform section 55 or the prediction error signal DS57supplied from the second inverse transform section 56. The arithmeticsection 57 stores the generated image signal DS58 in the referencememory 58.

The reference memory 58 stores the image signal of adjacent blocks andthe image signal of a plurality of frames. Incidentally, the imagesignal of the plurality of frames is stored after being subjected todeblocking filter processing.

The reference image signal DS60 of the adjacent blocks which imagesignal is read from the reference memory 58 is supplied to thepredicting section 60 and the image signal selecting switch 70. Inaddition, the reference image signal DS65 of the frame images whichimage signal is read from the reference memory 58 is supplied to themotion compensating section 61.

The predicting section 60 performs prediction in the prediction modeindicated by the prediction mode information DS53 using the referenceimage signal DS60 read from the reference memory 58, generates apredicted image signal DS61, and supplies the predicted image signalDS61 to the prediction selecting switch 62.

When an inter-frame prediction mode is selected as an optimum mode, themotion compensating section 61 performs motion compensation using thereference image signal DS65 on the basis of motion vector information inthe optimum mode, and generates a predicted image signal DS66. Themotion compensating section 61 supplies the generated predicted imagesignal DS66 to the prediction selecting switch 62 and the image signalselecting switch 70.

When an intra-frame prediction mode is selected as an optimum mode, theprediction selecting switch 62 selects the predicted image signal DS61generated in the predicting section 60, and outputs the predicted imagesignal DS61 as a predicted image signal DS73 to the arithmetic section57. When an inter-frame prediction mode is selected as an optimum mode,the prediction selecting switch 62 selects the predicted image signalDS66 generated in the motion compensating section 61, and outputs thepredicted image signal DS66 as the predicted image signal DS73 to thearithmetic section 57.

When an intra-frame prediction mode is selected as an optimum mode, theimage signal selecting switch 70 selects the reference image signalDS60, and outputs the reference image signal DS60 to the reference imageedge detecting section 71. When an inter-frame prediction mode isselected as an optimum mode, the image signal selecting switch 70selects the predicted image signal DS66 generated in the motioncompensating section 61, and outputs the predicted image signal DS66 tothe reference image edge detecting section 71.

The reference image edge detecting section 71 is configured in a similarmanner to the reference image edge detecting section 31 in the imagecoding device 10 b. The reference image edge detecting section 71detects an edge using the decoded image signal selected by the imagesignal selecting switch 70, and outputs an index DS75 indicating theposition of the edge and the intensity of the edge to the transformblock setting section 72.

The transform block setting section 72 is configured in a similar mannerto the transform block setting section 32 in the image coding device 10b. The transform block setting section 72 estimates the continuity ofthe edge within the sub-block as a coding object on the basis of theindex DS75 supplied from the reference image edge detecting section 71and the prediction mode information DS53 supplied from the entropydecoding section 51 b. The transform block setting section 72 setstransform blocks in an inverse orthogonal transform and dequantizationfrom a result of the estimation, and generates the transform blocksetting information DS76 indicating the set transform blocks. Thetransform block setting section 72 outputs the generated transform blocksetting information DS76 to the first dequantizing section 53, thesecond dequantizing section 54, the first inverse transform section 55,and the second inverse transform section 56.

The decoding control section 80 issues control instructions in theprocess of decoding the coded data and the like.

[3-4. Operation of Image Decoding Device]

FIG. 29 is a flowchart of operation of the image decoding device 50 b inthe third embodiment. In step ST151, the image decoding device 50 bobtains coded data. The image decoding device 50 b obtains the codeddata DSC, and starts decoding in each macroblock or each macroblockpair. The image decoding device 50 b then proceeds to step ST152.

In step ST152, the image decoding device 50 b performs entropy decoding.The image decoding device 50 b decodes the variable-length code of eachsyntax of the coded data DSC, and reproduces quantized data DS51,transform information DS52, prediction mode information DS53, and motionvector information DS35. The image decoding device 50 b then proceeds tostep ST153.

In step ST153, the image decoding device 50 b performs syntax analysis.The image decoding device 50 b analyzes the syntaxes from the dataobtained by performing the decoding. The image decoding device 50 b thenproceeds to step ST154.

In step ST154, the image decoding device 50 b performs initializationrelating to sub-blocks. The image decoding device 50 b initializes asub-block index sub_blk to “sub_blk=0” and sets a maximum sub-blocknumber MAX_SUB_BLK at the same time. The image decoding device 50 b thenproceeds to step ST155.

In step ST155, the image decoding device 50 b determines whether thesub-block index sub_blk is smaller than the maximum sub-block numberMAX_SUB_BLK. When the sub-block index sub_blk is smaller than themaximum sub-block number MAX_SUB_BLK, there is a sub-block yet to bedecoded among sub-blocks within a macroblock, and therefore the imagedecoding device 50 b proceeds to step ST156. When the sub-block indexsub_blk is not smaller than the maximum sub-block number MAX_SUB_BLK,there is no sub-block yet to be decoded among the sub-blocks within themacroblock, and therefore the image decoding device 50 b proceeds tostep ST170.

In step ST156, the image decoding device 50 b determines whetherinter-frame prediction is selected. When inter-frame prediction isselected as an optimum mode, the image decoding device 50 b proceeds tostep ST157. When intra-frame prediction is selected, the image decodingdevice 50 b proceeds to step ST169.

In step ST157, the image decoding device 50 b loads a predictiondirection index mode_idx_d, transform information trans_idx, and motionvector information. The image decoding device 50 b extracts theprediction direction index mode_idx_d, the transform informationtrans_idx, and the motion vector information from the coded data. Theimage decoding device 50 b then proceeds to step ST158.

In step ST158, the image decoding device 50 b generates amotion-compensated signal. The image decoding device 50 b generates themotion-compensated signal of search positions MV_x and MV_y indicated bythe motion vector of the motion vector information DS35 on the basis ofthe image signal of a reference frame and the motion vector informationDS35. The image decoding device 50 b then proceeds to step ST159.

In step ST159, the image decoding device 50 b detects an edge. The imagedecoding device 50 b detects an edge using the generatedmotion-compensated signal, and generates an index DS75 indicating theposition of the edge and the intensity of the edge. The image decodingdevice 50 b then proceeds to step ST160.

In step ST160, the image decoding device 50 b sets transform blocks. Theimage decoding device 50 b sets transform blocks from the continuity ofthe edge within the sub-block as a decoding object on the basis of theindex DS75 indicating the position of the edge and the intensity of theedge. The image decoding device 50 b then proceeds to step ST161.

In step ST161, the image decoding device 50 b determines whether theindex mode_idx_d is smaller than a minimum mode number mode_direction_dof oblique direction prediction modes as a minimum value of the modenumbers of the oblique prediction direction modes, or whether thetransform information is “trans_idx=1.” In at least one of the caseswhere the prediction direction index mode_idx_d is smaller than theminimum mode number mode_direction_d and where the transform informationis “trans_idx=1,” the image decoding device 50 b proceeds to step ST162.Otherwise, the image decoding device 50 b proceeds to step ST164.

In step ST162, the image decoding device 50 b performs horizontal andvertical dequantization. The image decoding device 50 b then proceeds tostep ST163. In step ST163, the image decoding device 50 b performs ahorizontal and vertical inverse DCT. The image decoding device 50 b thenproceeds to step ST166. The image decoding device 50 b for examplechanges the processing selecting switch 52 to the side of the firstdequantizing section 53, and performs the dequantization and the inverseDCT using the first dequantizing section 53 and the first inversetransform section 55.

In step ST164, the image decoding device 50 b performs oblique directionpattern dequantization. The image decoding device 50 b then proceeds tostep ST165. In step ST165, the image decoding device 50 b performs anoblique direction pattern inverse DCT. The image decoding device 50 bthen proceeds to step ST166. The image decoding device 50 b for examplechanges the processing selecting switch 52 to the side of the seconddequantizing section 54. In addition, the image decoding device 50 bchanges pattern selecting switches in the second dequantizing section 54and the second inverse transform section 56 according to the predictiondirection index mode_idx_d. By changing the switches according to theindex mode_idx_d, the image decoding device 50 b performs thedequantization and the inverse DCT using an oblique direction patterndequantizing section and an oblique direction pattern inverse DCTsection corresponding to the prediction direction.

In step ST166, the image decoding device 50 b synthesizes a predictionerror and a predicted image. The image decoding device 50 b generates animage signal DS58 by adding a predicted image signal DS73 supplied fromthe prediction selecting switch 62 to a prediction error signal DS56 ora prediction error signal DS57. The image decoding device 50 b thenproceeds to step ST167.

In step ST167, the image decoding device 50 b stores the generated imagesignal DS58 in the reference memory. The image decoding device 50 bstores the generated image signal DS58 in the reference memory 58. Theimage decoding device 50 b then proceeds to step ST168.

In step ST168, the image decoding device 50 b adds “1” to the sub-blockindex sub_blk to set a new index sub_blk. The image decoding device 50 bthen returns to step ST155.

When determining in step ST156 that intra-frame prediction is selectedand thus proceeding to step ST169, the image decoding device 50 bperforms a process from step ST56 to step ST67 in FIG. 20. The imagedecoding device 50 b then returns to step ST155.

Thereafter, the image decoding device 50 b determines in step ST155 thatthe sub-block index sub_blk is not smaller than the maximum sub-blocknumber MAX_SUB_BLK. The image decoding device 50 b then proceeds to stepST170. In step ST170, the image decoding device 50 b outputs an imagesignal stored in the reference memory 58 after completion of thedecoding of the sub-blocks as the image signal of a decoded image.

Thus, according to the third embodiment, transform blocks are set inconsideration of edge continuity on the basis of not only intra-frameprediction but also inter-frame prediction. Thus, also in inter-frameprediction, it is possible to prevent a continuous edge from straddlinga plurality of transform blocks and thus improve subjective imagequality. In addition, transform blocks not including an edge areincreased, so that energy concentration can be improved.

Incidentally, the third embodiment calculates coding cost for eachprediction direction while changing the search positions in order.However, it is also possible for example to detect a block positionhaving highest correlation with a sub-block as a coding object,calculate coding cost for each prediction direction at the blockposition having the highest correlation, compare the calculated codingcost for each prediction direction with coding cost for each predictionmode of intra-frame prediction, and determine an optimum mode. In thiscase, operation processing can be reduced because there is no need tocalculate coding cost each time the search positions are changed inorder.

In addition, the series of processes described in the specification canbe performed by hardware, software, or a composite configuration of bothhardware and software. When processing is performed by software, aprogram in which a processing sequence is recorded is executed afterbeing installed into a memory within a computer incorporated indedicated hardware. Alternatively, the program can be executed afterbeing installed on a general-purpose computer capable of performingvarious kinds of processing.

For example, the program can be recorded on a hard disk or a ROM (ReadOnly Memory) as a recording medium in advance. Alternatively, theprogram can be stored (recorded) temporarily or permanently on aremovable recording medium such as a flexible disk, a CD-ROM (CompactDisc Read Only Memory), an MO (Magneto-Optical) disk, a DVD (DigitalVersatile Disc), a magnetic disk, a semiconductor memory, or the like.Such a removable recording medium can be provided as so-called packagedsoftware.

Incidentally, in addition to being installed from a removable recordingmedium as described above onto a computer, the program is transferred byradio from a download site to a computer or transferred by wire to acomputer via networks such as a LAN (Local Area Network), the Internet,and the like. The computer can receive the program transferred in such amanner, and install the program onto a recording medium such as abuilt-in hard disk or the like.

It is to be noted that the various kinds of processing described in thespecification may be not only performed in time series according to thedescription but also performed in parallel or individually according tothe processing power of a device performing the processing or accordingto necessity. In addition, a system in the present specification is alogical set configuration of a plurality of devices, and is not limitedto a system in which devices of respective configurations are presentwithin an identical casing.

In addition, the foregoing embodiments use a DCT as an orthogonaltransform method, but may use a KLT (Karhunen-Loeve transform), a DST(discrete sine transform), or a DWT (discrete wavelet transform). Inaddition, block size is not limited to sizes in the foregoingembodiments, but may be larger block sizes.

The foregoing embodiments disclose the present technology in anillustrative form. It is obvious that modifications and substitutions inthe embodiments can be made by those skilled in the art withoutdeparting from the spirit of the present technology. That is, in orderto determine the spirit of the present technology, claims are to beconsidered.

According to an image coding device, an image decoding device, methodsthereof, and programs according to an embodiment of the presenttechnology, an edge is detected using the image signal of a referenceimage for a coding object block. On the basis of a result of the edgedetection, transform blocks are set by dividing the coding object blocksuch that boundaries between the blocks after the division do notinclude the edge. In addition, transform processing is performed foreach transform block, and coded data is generated. The image decodingdevice decoding the coded data detects the edge using the image signalof the reference image for a decoding object block. On the basis of aresult of the edge detection, transform blocks are set by dividing thedecoding object block such that boundaries between the blocks after thedivision do not include the edge. In addition, inverse transformprocessing is performed for each transform block, and the image signalof a decoded image is generated. It is therefore possible to prevent acontinuous edge from straddling a plurality of transform blocks and thusimprove subjective image quality. In addition, transform blocks notincluding an edge can be increased, so that an effect of improvingefficiency of energy concentration can be obtained. Thus, the presenttechnology is suitable for imaging devices that generate moving imagesand still images, editing devices that edit moving images and stillimages, and the like.

The present application contains subject matter related to thatdisclosed in Japanese Priority Patent Application JP 2011-027385 filedin the Japan Patent Office on Feb. 10, 2011, the entire content of whichis hereby incorporated by reference.

What is claimed is:
 1. An image coding device comprising: an edgedetecting section configured to perform edge detection using an imagesignal of a reference image for a coding object block; a transform blocksetting section configured to set transform blocks by dividing saidcoding object block such that a boundary between the blocks afterdivision does not include an edge on a basis of a result of said edgedetection; and a coding processing section configured to generate codeddata by performing processing including an orthogonal transform of eachof said transform blocks.
 2. The image coding device according to claim1, wherein said transform block setting section estimates continuity ofthe edge in the coding object block on the basis of the result of saidedge detection, and sets said transform blocks by dividing said codingobject block such that the boundary between the blocks does not includethe estimated edge.
 3. The image coding device according to claim 2,wherein said transform block setting section sets said transform blocksfor each prediction mode of one of inter-frame prediction andintra-frame prediction, and said coding processing section comparescoding cost when the processing including said orthogonal transform isperformed in the transform blocks set for each said prediction mode, andselects a prediction mode of highest coding efficiency.
 4. The imagecoding device according to claim 3, wherein said edge detecting sectionuses an image signal of a coded block adjacent to said coding objectblock as the image signal of said reference image, and said transformblock setting section estimates that the edge detected by said edgedetection is continuous in a prediction direction of a prediction mode,and sets said transform blocks.
 5. The image coding device according toclaim 3, wherein said edge detecting section uses an image signal of acoded image in a temporal direction with respect to said coding objectblock as the image signal of said reference image.
 6. The image codingdevice according to claim 3, wherein said coding processing sectionincludes information indicating the selected prediction mode in thecoded data obtained by performing coding processing in said selectedprediction mode.
 7. The image coding device according to claim 3,wherein said coding processing section includes information indicatingtransform blocks set in the selected prediction mode in the coded dataobtained by performing coding processing in said selected predictionmode.
 8. The image coding device according to claim 2, wherein when aposition of the edge and a direction of the edge are detected by saidedge detection, said transform block setting section comparesreliability of a prediction direction of a prediction mode withreliability of the detected direction of said edge, and estimates thatthe edge is continuous in a direction of higher reliability.
 9. Theimage coding device according to claim 1, wherein said edge detectingsection detects intensity of the edge by said edge detection, and saidtransform block setting section selects the edge not to be included inthe boundary between said transform blocks according to the intensity ofsaid edge.
 10. An image coding method comprising: performing edgedetection using an image signal of a reference image for a coding objectblock; setting transform blocks by dividing said coding object blocksuch that a boundary between the blocks after division does not includean edge on a basis of a result of said edge detection; and generatingcoded data by performing processing including an orthogonal transform ofeach of said transform blocks.
 11. A non-transitory computer-readablemedium storing computer-readable instructions thereon which whenexecuted by a computer cause the computer to perform a methodcomprising: performing edge detection using an image signal of areference image for a coding object block; setting transform blocks bydividing said coding object block such that a boundary between theblocks after division does not include an edge on a basis of a result ofsaid edge detection; and generating coded data by performing processingincluding an orthogonal transform of each of said transform blocks. 12.An image decoding device comprising: an information extracting sectionconfigured to extract prediction mode information from coded data; anedge detecting section configured to perform edge detection using animage signal of a reference image for a decoding object block; atransform block setting section configured to set transform blocks bydividing said decoding object block such that a boundary between theblocks after division does not include an edge on a basis of saidprediction mode information and a result of said edge detection; and adecoding processing section configured to generate an image signal byperforming processing including an inverse orthogonal transform of eachof said transform blocks.
 13. The image decoding device according toclaim 12, wherein said transform block setting section estimatescontinuity of the edge in the decoding object block on the basis of theresult of said edge detection, and sets said transform blocks bydividing said decoding object block such that the boundary between theblocks does not include the estimated edge.
 14. The image decodingdevice according to claim 13, wherein said edge detecting section usesan image signal of a decoded block adjacent to said decoding objectblock as the image signal of said reference image, and said transformblock setting section estimates that the edge detected by said edgedetection is continuous in a prediction direction of a prediction modeindicated by said prediction mode information, and sets said transformblocks.
 15. The image decoding device according to claim 13, whereinsaid edge detecting section uses an image signal of a decoded image in atemporal direction with respect to said decoding object block as theimage signal of said reference image.
 16. The image decoding deviceaccording to claim 13, wherein when a position of the edge and adirection of the edge are detected by said edge detection, saidtransform block setting section compares reliability of a predictiondirection of a prediction mode with reliability of the detecteddirection of said edge, and estimates that the edge is continuous in adirection of higher reliability.
 17. The image decoding device accordingto claim 12, wherein said edge detecting section detects intensity ofthe edge by said edge detection, and said transform block settingsection selects the edge not to be included in the boundary between saidtransform blocks according to the intensity of said edge.
 18. An imagedecoding method comprising: extracting prediction mode information fromcoded data; performing edge detection using an image signal of areference image for a decoding object block; setting transform blocks bydividing said decoding object block such that a boundary between theblocks after division does not include an edge on a basis of saidprediction mode information and a result of said edge detection; andgenerating an image signal by performing processing including an inverseorthogonal transform of each of said transform blocks.
 19. Anon-transitory computer-readable medium storing computer-readableinstructions thereon which when executed by a computer cause thecomputer to perform a method comprising: extracting prediction modeinformation from said coded data; performing edge detection using animage signal of a reference image for a decoding object block; settingtransform blocks by dividing said decoding object block such that aboundary between the blocks after division does not include an edge on abasis of said prediction mode information and a result of said edgedetection; and generating an image signal by performing processingincluding an inverse orthogonal transform of each of said transformblocks.